TIL: Configuring multiple FLAME backends

TIL: Configuring multiple FLAME backends

In OctoCut, I have to perform a few tasks that are CPU or memory intensive. Instead of performing them on the machine serving web requests, I want to perform them on a different machine. FLAME is perfect for this. (If you're not familiar with FLAME, I highly recommend checking out the original announcement video.)

However, there are two different kinds of tasks and I wanted them to be assigned to different kinds of machines. You can create multiple FLAME pools, but the FLAME docs only describe how to globally configure a single backend.

Here's how I configured different backends for multiple pools:

# config.exs

  config :flame,
         :backend_a,
         {FLAME.FlyBackend,
          cpu_kind: "performance",
          cpus: 4,
          memory_mb: 8192,
          token: System.fetch_env!("FLY_API_TOKEN")}

  config :flame,
         :backend_b,
         {FLAME.FlyBackend,
          cpu_kind: "shared", 
          cpus: 2, 
          memory_mb: 4096, 
          token: System.fetch_env!("FLY_API_TOKEN")}

And in my application.ex's list of children to start, I used the backends we defined when specifying the FLAME pools:

children = [
        # ...
        # This pool runs more memory/CPU-intensive tasks
        {FLAME.Pool,
         name: OctoCut.PoolA,
         # ... other FLAME.Pool options
         backend: Application.get_env(:flame, :backend_a)},
         
        # This one is doing the "easy" tasks
        {FLAME.Pool,
         name: OctoCut.PoolB,
         # ... other FLAME.Pool options
         backend: Application.get_env(:flame, :backend_b)},
         
         ]