Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [celery]

Celery is a distributed task queue written in Python.

35 questions
1
vote
1 answer

AWS autoscaling: How to kill the machines that are not busy

We have an autoscaling group that spawns worker servers. Each worker server runs celery processes. We monitor the celery queue length using cloudwatch and depending on that queue length we spawn/kill auto scaling server. What you see in this answer…
Ranjith Ramachandra
  • 425
  • 4
  • 13
1
vote
1 answer

ubuntu grant user permissions to run file without SU - running python celery

im having trouble with running celery as a daemon on ubuntu. I have run the debug command as suggested in the documentation http://docs.celeryproject.org/en/latest/userguide/daemonizing.html#example-configuration (troubleshooting section) sh -x…
AlexW
  • 13
  • 2
1
vote
0 answers

Rabbitmq running out of memory due to "Processes / other"

I am running rabbitmq within kubernetes using the rabbitmq:3.6.11-management docker image (https://hub.docker.com/_/rabbitmq/). Looking at the management dashboard, there seems to be a lot of memory being used by the "Processes / other" category…
JBBreeman
  • 11
  • 1
1
vote
0 answers

How to view pending messages in RabbitMQ admin?

How do you view pending/queued messages in RabbitMQ admin? I have the RabbitMQ admin tool running, and the overview page is showing that there are over 4000 messages ready. It's also showing that there are over 90 consumers. Yet the number of ready…
Cerin
  • 3,497
  • 17
  • 57
  • 72
1
vote
1 answer

DatabaseError between Oracle and Django with Celery

I'm having trouble with what seems to be the very last step in the deployment of a Django-Celery-RabbitMQ project. After running the celery, httpd and rabbitmq-server services through systemd, when I trigger a task in Django I see the message…
vowel-house-might
  • 259
  • 1
  • 3
  • 11
1
vote
1 answer

How to stop celery worker?

I am running celery via redis. I start my worker like this: celery multi start worker1 -A mypackage.tasks.tasks celery multi v3.1.18 (Cipater) > Starting nodes... > worker1@polyphemus.xxx.net: OK And I kill it like this: celery multi kill…
blueFast
  • 4,000
  • 13
  • 36
  • 51
1
vote
0 answers

Supervisor start/stop/restart not behaving as expected

I am using supervisor to manage celery and due to occasional project requirements am required to add environment variables. I set these in the supervisor [environment] section of the .conf file but simply using service supervisor stop service…
bgrantdev
  • 111
  • 1
  • 3
1
vote
1 answer

Supervisord as Windows Service on Cygwin

I am attempting to run Celery as a Windows Service using Supervisord. I followed the configuration laid out on the Celery site and here. I have set up a virtual environment to run supervisord through cygwin. I have highlighted the lines I think are…
Jason Hawkins
  • 111
  • 5
1
vote
1 answer

Celery and RabbitMQ - Tools for monitoring and callout alarms

I think we've reached the limit of usability of celery events and celery fllower. Is there a good tool for monitoring status of Celery running under RabbitMQ? In particular, I would like something to raise an alarm when: My queues are backing up…
Goro
  • 654
  • 3
  • 9
  • 18
1
vote
0 answers

OSError : [Errno 38] Function not implemented - Django Celery implementation

I installed django-celery and I tried to start up the worker server but I get an OSError that a function isn't implemented. I'm running CentOS release 5.4 (Final) on a VPS: . broker -> amqp://guest@localhost:5672/ . queues -> .…
Jordan Messina
  • 111
  • 5
0
votes
1 answer

Remove consumer before start deployment from RabbitMQ queue

We have setup of Celery worker with 8 node setup. It create 8 queues in RabbitMQ. When we start deploying new changes, last step in ansible playbook is Celery restart. Celery restart has to down each node and start that node. But, there are continue…
Nilesh
  • 255
  • 1
  • 6
  • 17
0
votes
2 answers

What is the correct syntax to run cron every 48 hours?

Will this work correctly? CELERYBEAT_SCHEDULE = { 'task-number-one': { 'task': 'project.users.tasks.send_something', 'schedule': crontab(minute='*/2880'), } }
paolo
  • 101
  • 1
  • 1
0
votes
1 answer

Graceful shutdown of celery workers with systemd?

I have a box with celery workers, some of which perform reasonably long-running tasks (a few minutes). When this box updates, I'd like to shutdown the celery workers gracefully, so that ideally: New tasks stop being sent to workers Existing tasks…
user31415629
  • 301
  • 2
  • 12
0
votes
1 answer

How to get logwatch to filter dates in celery logs?

I'm a newbie to Logwatch and have it set up to watch log files created by celery, a distributed task queue popular in the Python world. My logwatch.conf "Range" is set to "between yesterday 4am and today 4am” and this range filter works for other…
Ghopper21
  • 113
  • 9
0
votes
0 answers

Run shell script as Supervisord subprocess?

I have a Django 1.6.2 application that uses Celery 3.1.7 for asynchronous tasks. I'm starting my Celery workers using Supervisor. So far everything is working well except when I reboot my Debian 7.8 server. When that happens, my Celery workers…
Jim
  • 320
  • 7
  • 16
1
2
3