Oracle 和 Django 之間的 DatabaseError 與 Celery
我在部署 Django-Celery-RabbitMQ 項目的最後一步時遇到了麻煩。
通過 systemd 執行 celery、httpd 和 rabbitmq-server 服務後,當我在 Django 中觸發任務時,我看到了消息
Received task: smartalec.tasks.match_loadfile_task
smartalec 是我的應用程序,match_loadfile_task 是我的任務,該項目稱為 tcida_api。
但是大約 30 秒後,有一個巨大的堆棧跟踪,如下所示(完整的調試日誌,下面有異常)。
[2016-09-09 15:34:20,040: ERROR/MainProcess] Task smartalec.tasks.match_loadfile_task[740dc792-7410-4418-aa44-dd9c56b63b04] raised unexpected: DatabaseError(' <cx_Oracle._Error object at 0x3a3ed50>',)
Google搜尋“Django Oracle Celery DatabaseError”會得到一些關於在單個連接上共享執行緒的查詢,但我不確定這是否適用於我,因為我在 project.settings 的數據庫配置中有“threaded=TRUE”。
當我從命令行執行 celery 時,我得到了同樣的結果。
sudo /opt/tcida/virtualenv/smartalec/bin/celery -A my_django_project worker -l info
我沒有安裝 django celery,django 可以讓數據庫查詢沒問題,通過 shell 和網頁,配置在我的本地機器上使用 django 的開發伺服器。
什麼會導致或解決此異常?
完整的 Celery 調試日誌,不包括完整的堆棧跟踪。
[amartin@ip-10-50-7-148 tcida_api]$ sudo /opt/tcida/virtualenv/smartalec/bin/celery -A tcida_api worker --concurrency 1 -l debug /opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/celery/platforms.py:812: RuntimeWarning: You are running the worker with superuser privileges, which is absolutely not recommended! Please specify a different user using the -u option. User information: uid=0 euid=0 gid=0 egid=0 uid=uid, euid=euid, gid=gid, egid=egid, [2016-09-09 16:14:22,995: DEBUG/MainProcess] | Worker: Preparing bootsteps. [2016-09-09 16:14:22,997: DEBUG/MainProcess] | Worker: Building graph... [2016-09-09 16:14:22,997: DEBUG/MainProcess] | Worker: New boot order: {Timer, Hub, Queues (intra), Pool, Autoscaler, StateDB, Autoreloader, Beat, Consumer} [2016-09-09 16:14:22,999: DEBUG/MainProcess] | Consumer: Preparing bootsteps. [2016-09-09 16:14:23,000: DEBUG/MainProcess] | Consumer: Building graph... [2016-09-09 16:14:23,003: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Tasks, Control, Agent, Gossip, Heart, event loop} -------------- celery@ip-10-50-7-148.eu-west-1.compute.internal v3.1.23 (Cipater) ---- **** ----- --- * *** * -- Linux-3.10.0-229.11.1.el7.x86_64-x86_64-with-redhat-7.1-Maipo -- * - **** --- - ** ---------- [config] - ** ---------- .> app: tcida_api:0x1a91850 - ** ---------- .> transport: amqp://guest:**@localhost:5672// - ** ---------- .> results: disabled:// - *** --- * --- .> concurrency: 1 (prefork) -- ******* ---- --- ***** ----- [queues] -------------- .> celery exchange=celery(direct) key=celery [tasks] . celery.backend_cleanup . celery.chain . celery.chord . celery.chord_unlock . celery.chunks . celery.group . celery.map . celery.starmap . smartalec.tasks.match_loadfile_task [2016-09-09 16:14:23,005: DEBUG/MainProcess] | Worker: Starting Hub [2016-09-09 16:14:23,005: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:23,005: DEBUG/MainProcess] | Worker: Starting Pool [2016-09-09 16:14:23,029: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:23,030: DEBUG/MainProcess] | Worker: Starting Consumer [2016-09-09 16:14:23,030: DEBUG/MainProcess] | Consumer: Starting Connection [2016-09-09 16:14:23,038: DEBUG/MainProcess] Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'consumer_cancel_notify': True, u'publisher_confirms': True, u'basic.nack': True}, u'platform': u'Erlang/OTP', u'version': u'3.1.5'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US'] [2016-09-09 16:14:23,039: DEBUG/MainProcess] Open OK! [2016-09-09 16:14:23,039: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672// [2016-09-09 16:14:23,040: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:23,040: DEBUG/MainProcess] | Consumer: Starting Events [2016-09-09 16:14:23,047: DEBUG/MainProcess] Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2013 GoPivotal, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'consumer_cancel_notify': True, u'publisher_confirms': True, u'basic.nack': True}, u'platform': u'Erlang/OTP', u'version': u'3.1.5'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US'] [2016-09-09 16:14:23,047: DEBUG/MainProcess] Open OK! [2016-09-09 16:14:23,048: DEBUG/MainProcess] using channel_id: 1 [2016-09-09 16:14:23,048: DEBUG/MainProcess] Channel open [2016-09-09 16:14:23,049: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:23,049: DEBUG/MainProcess] | Consumer: Starting Mingle [2016-09-09 16:14:23,049: INFO/MainProcess] mingle: searching for neighbors [2016-09-09 16:14:23,050: DEBUG/MainProcess] using channel_id: 1 [2016-09-09 16:14:23,050: DEBUG/MainProcess] Channel open [2016-09-09 16:14:24,056: INFO/MainProcess] mingle: all alone [2016-09-09 16:14:24,057: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:24,057: DEBUG/MainProcess] | Consumer: Starting Tasks [2016-09-09 16:14:24,059: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:24,059: DEBUG/MainProcess] | Consumer: Starting Control [2016-09-09 16:14:24,059: DEBUG/MainProcess] using channel_id: 2 [2016-09-09 16:14:24,060: DEBUG/MainProcess] Channel open [2016-09-09 16:14:24,062: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:24,062: DEBUG/MainProcess] | Consumer: Starting Gossip [2016-09-09 16:14:24,062: DEBUG/MainProcess] using channel_id: 3 [2016-09-09 16:14:24,063: DEBUG/MainProcess] Channel open [2016-09-09 16:14:24,065: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:24,065: DEBUG/MainProcess] | Consumer: Starting Heart [2016-09-09 16:14:24,066: DEBUG/MainProcess] ^-- substep ok [2016-09-09 16:14:24,066: DEBUG/MainProcess] | Consumer: Starting event loop [2016-09-09 16:14:24,067: WARNING/MainProcess] celery@ip-10-50-7-148.eu-west-1.compute.internal ready. [2016-09-09 16:14:24,067: DEBUG/MainProcess] | Worker: Hub.register Pool... [2016-09-09 16:14:24,067: DEBUG/MainProcess] basic.qos: prefetch_count->4 [2016-09-09 16:14:31,300: INFO/MainProcess] Received task: smartalec.tasks.match_loadfile_task[51fed74e-f0ce-4cad-9a3d-283b830939d6] [2016-09-09 16:14:31,300: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x1c50500> (args:(u'smartalec.tasks.match_loadfile_task', u'51fed74e-f0ce-4cad-9a3d-283b830939d6', [1], {}, {u'utc': True, u'is_eager': False, u'chord': None, u'group': None, u'args': [1], u'retries': 0, u'delivery_info': {u'priority': 0, u'redelivered': False, u'routing_key': u'celery', u'exchange': u'celery'}, u'expires': None, u'hostname': 'celery@ip-10-50-7-148.eu-west-1.compute.internal', u'task': u'smartalec.tasks.match_loadfile_task', u'callbacks': None, u'correlation_id': u'51fed74e-f0ce-4cad-9a3d-283b830939d6', u'errbacks': None, u'timelimit': [None, None], u'taskset': None, u'kwargs': {}, u'eta': None, u'reply_to': u'950dab7a-8608-37e0-991a-5e7c6dadbf4a', u'id': u'51fed74e-f0ce-4cad-9a3d-283b830939d6', u'headers': {}}) kwargs:{}) [2016-09-09 16:14:31,310: DEBUG/MainProcess] Task accepted: smartalec.tasks.match_loadfile_task[51fed74e-f0ce-4cad-9a3d-283b830939d6] pid:25239 [2016-09-09 16:15:26,475: ERROR/MainProcess] Task smartalec.tasks.match_loadfile_task[51fed74e-f0ce-4cad-9a3d-283b830939d6] raised unexpected: DatabaseError('<cx_Oracle._Error object at 0x36c9d50>',) Traceback (most recent call last): File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task R = retval = fun(*args, **kwargs) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__ return self.run(*args, **kwargs) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/smartalec/tasks.py", line 14, in match_loadfile_task load_file = LoadFile.objects.get(pk=loadfile_pk) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/query.py", line 379, in get num = len(clone) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/query.py", line 238, in __len__ self._fetch_all() File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/query.py", line 1087, in _fetch_all self._result_cache = list(self.iterator()) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/query.py", line 54, in __iter__ results = compiler.execute_sql() File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 824, in execute_sql sql, params = self.as_sql() File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/oracle/compiler.py", line 22, in as_sql subquery=subquery, File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 376, in as_sql where, w_params = self.compile(self.where) if self.where is not None else ("", []) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 353, in compile sql, params = node.as_sql(self, self.connection) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/sql/where.py", line 79, in as_sql sql, params = compiler.compile(child) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 353, in compile sql, params = node.as_sql(self, self.connection) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/lookups.py", line 158, in as_sql rhs_sql = self.get_rhs_op(connection, rhs_sql) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/models/lookups.py", line 162, in get_rhs_op return connection.operators[self.lookup_name] % rhs File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/oracle/base.py", line 80, in __get__ instance.cursor().close() File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/base/base.py", line 233, in cursor cursor = self.make_cursor(self._cursor()) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/base/base.py", line 204, in _cursor self.ensure_connection() File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/base/base.py", line 199, in ensure_connection self.connect() File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/base/base.py", line 199, in ensure_connection self.connect() File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/base/base.py", line 171, in connect self.connection = self.get_new_connection(conn_params) File "/opt/tcida/virtualenv/smartalec/lib/python2.7/site-packages/django/db/backends/oracle/base.py", line 216, in get_new_connection return Database.connect(conn_string, **conn_params) DatabaseError: <cx_Oracle._Error object at 0x36c9d50>
確保執行 celery 的使用者(應該是帶有 shell 的使用者,按照慣例稱為
celery
)已TNS_ADMIN
在其環境中設置。如果您通過 systemd 執行 celery 服務,那麼它將由一個名為 的文件
celery.service
和/etc/systemd/system/celery.service
權限 644 來控制。在該[Service]
部分下的該文件中,使用其中一個指令Environment
直接設置環境文件或EnvironmentFile
使用環境變數文件。還請務必了解 services
celery
和之間的區別celeryd
,我自己並不真正知道它是什麼,但它們不能像您/我所做的那樣互換使用。