Multiple girder instances

Support
1

Hi,

I have 2 Girders running on the same machine (sharing the same database but having different configuration and behavior).
Girder workers can be triggered by any Girder instance. I can not have a unique celery queue because I get 401 HTTP errors (authentification on Girder #2 is different from Girder#1) How can I have 2 instances of celery running and making sure they consume their own “jobs” ? Should I create 2 queues, if so, how ?

Thanks,
Julien.

2

I don’t think what you want to do is supported “out of the box”. It will likely involve writing a custom plugin to solve the problem.

I can think of a few solutions:

  • Inject some configuration to the celery app to route tasks to different queues per instance.
  • A custom girder plugin that overrides the worker.api_url setting per instance so they worker’s communicate back with the correct instance.
  • A custom girder plugin that overrides the broker setting per instance and maintain two independent brokers.