multithreading - How to execute multiple django celery tasks with a single worker? -

-

here celery file:

from __future__ import absolute_import import os celery import celery   # set default django settings module 'celery' program. os.environ.setdefault('django_settings_module', 'ets.settings')  django.conf import settings  # noqa  app = celery('proj',               broker='redis://myredishost:6379/0',              backend='redis://myredishost:6379/0',              include=['tracking.tasks'])  # optional configuration, see application user guide. app.conf.update(     celery_task_result_expires=3600, )  if __name__ == '__main__':     app.start()

here task file:

@app.task def escalate_to_sup(id, group):     escalation_email, created = escalationemail.objects.get_or_create()     escalation_email.send()      return 'sup email sent to: '+str(group)   @app.task def escalate_to_fm(id, group):     escalation_email, created = escalationemail.objects.get_or_create()     escalation_email.send()      return 'fm email sent to: '+str(group)

i start worker this:

celery -a ets worker -l info

i have tried add concurrency this:

celery -a ets worker -l info --concurrency=10

i attempt call tasks above following:

from tracking.tasks import escalate_to_fm, escalate_to_sup

def status_change(equipment):         r1 = escalate_to_sup.apply_async((equipment.id, [1,2]), countdown=10)         r2 = escalate_to_fm.apply_async((equipment.id, [3,4]), countdown=20)          print r1.id         print r2.id

this prints:

c2098768-61fb-41a7-80a2-f79a73570966 23959fa3-7f80-4e20-a42f-eef75e9bedeb

the escalate_to_sup , escalate_fm functions log worker intermittently. @ least 1 executes, never both.

i have tried spinning more workers, , both tasks execute. like:

celery -a ets worker -l info --concurrency=10 -n worker1.%h celery -a ets worker -l info --concurrency=10 -n worker2.%h

the problem don't know how many of tasks might execute concurrently spinning worker every possible tasks execute not feasible.

does celery expect work every active task?

how execute multiple tasks single worker?


Comments

Post a Comment

Popular posts from this blog

Django REST Framework perform_create: You cannot call `.save()` after accessing `serializer.data` -

-
i have following viewset: class artistprofileviewset(viewsets.modelviewset): queryset = artistprofile.objects.all() serializer_class = artistprofileserializer def perform_create(self, serializer): if serializer.is_valid(): serializer.save() with following serializers: class simpleartisttrackserializer(serializers.hyperlinkedmodelserializer): class meta: model = artisttrack fields = (...my fields...) class artistprofileserializer(serializers.hyperlinkedmodelserializer): owners = userserializer(many=true, required=false) tracks = simpleartisttrackserializer(many=true, required=false) class meta: model = artistprofile fields = (...my fields...) i getting following error: assertionerror: cannot call `.save()` after accessing `serializer.data`.if need access data before committing database inspect 'serializer.validated_data' instead. i don't see editing serializer.data object. th...
Read more

Why does Go error when trying to marshal this JSON? -

-
i'm trying rather simple example of trying decode json file structs using golang. however, when attempting error that json: cannot unmarshal object go value of type []main.track which don't understand. here's code in question, , json can found here . package main import ( "encoding/json" "fmt" "net/http" "os" ) // tracks group of track types type tracks struct { tracks []track } // track represents singular track type track struct { name string url string artist artist } // artist represents single artist type artist struct { name string mbid string url string } func main() { url := "http://ws.audioscrobbler.com/2.0/?method=geo.gettoptracks&api_key=c1572082105bd40d247836b5c1819623&format=json&country=netherlands" response, err := http.get(url) if err != nil { fmt.printf("%s\n", err) os.exit(1) } de...
Read more