Eventlet

I am writing a website for flask. There is a need to use Apsheduler for periodic task execution. But the problem is that the task is not executed periodically due to a conflict with eventlet. On the forums, I found that you need to wrap ... [More] the blocking construct in eventlet.tpool.execute(). Tell me what exactly do I need to wrap? This is how I create an instance of BackgroundScheduler scheduler = BackgroundScheduler(daemon=True) There is a class for working with tasks. from app import scheduler import json class SchedulerTask(object): schedulers_list_publish = None def __init__(self): self.schedulers_list_publish = list() def add_scheduler_publish(self, dev_id, mqtt, topic_req_res, m_req_state, qos_req, timer): id_sch = dev_id + "_scheduler" sc = scheduler.add_job(self.publish_async, args=[mqtt, topic_req_res, m_req_state, qos_req, timer, id_sch], id=id_sch, trigger='interval', seconds=timer) print(sc.id) self.schedulers_list_publish.append(id_sch) return id_sch def start_schedulers(self): print(self.schedulers_list_publish) scheduler.start() @staticmethod def del_schedulers(s_id): scheduler.remove_job(s_id) # передача запроса на получение данных @staticmethod def publish_async(mqtt, topic_req_res, m_req_state, qos_req, timer, id_sch): try: msg = json.dumps(m_req_state) mqtt.publish(topic_req_res, msg, qos_req) except Exception as ex: print("Error publish: " + str(ex)) I call from a function from another module: def _handle_connect(self, client, userdata, flags, rc): code_list = list() for dev in self.devices: if dev.device_code not in code_list: # запущен ли уже поток с таким кодом code_list.append(dev.device_code) mqtt.subscribe("BK" + dev.device_code + self.type_topic[1], self.qos_sub) self.schedulers_list.append(tpool.execute(sch_task.add_scheduler_publish, dev.device_code, mqtt, "BK" + dev.device_code + self.type_topic[0], self.m_request_state, self.qos_request, self.POOL_TIME)) sch_task.start_schedulers() [Less]

Currently, some of our backend functions overlap with each other in terms of db writes, and we have eventlet money patch spawning multiple threads, so we have to lock. We'd like this lock to be per a column value (i.e. for every unique ... [More] value in some column, we want a different lock). We have tried a python primitive ilock that someone created on Github (https://github.com/symonsoft/ilock), but it relies on creating a local file, so it doesn't work as we scale up across machines. Is there a way to lock functions and store that lock on the database, and not on the local machine? [Less]

I'm using Flask-SocketIO for a project of mine, and it's causing some issues, i.e. the client isn't able to connect to the server. I figured this may be due to it using the flask default development server, so I tried to install ... [More] eventlet, but it didn't work. I got this error: Running setup.py install for greenlet ... error ERROR: Command errored out with exit status 1: command: /Users/arnu515/Documents/code/videochat/temp/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/62/291hqhv517j2x50bfw65mj8r0000gn/T/pip-install-kilo4pkb/greenlet/setup.py'"'"'; __file__='"'"'/private/var/folders/62/291hqhv517j2x50bfw65mj8r0000gn/T/pip-install-kilo4pkb/greenlet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/62/291hqhv517j2x50bfw65mj8r0000gn/T/pip-record-bl7qcak2/install-record.txt --single-version-externally-managed --compile --install-headers /Users/arnu515/Documents/code/videochat/temp/venv/include/site/python3.8/greenlet cwd: /private/var/folders/62/291hqhv517j2x50bfw65mj8r0000gn/T/pip-install-kilo4pkb/greenlet/ Complete output (112 lines): running install running build running build_ext building 'greenlet' extension creating build creating build/temp.macosx-10.14.6-x86_64-3.8 xcrun -sdk macosx clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -I/Users/arnu515/Documents/code/videochat/temp/venv/include -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c greenlet.c -o build/temp.macosx-10.14.6-x86_64-3.8/greenlet.o In file included from greenlet.c:5: In file included from ./greenlet.h:8: In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11: In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/limits.h:21: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h:63: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture #error Unsupported architecture ^ In file included from greenlet.c:5: In file included from ./greenlet.h:8: In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11: In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/limits.h:21: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h:64: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/limits.h:8:2: error: architecture not supported #error architecture not supported ^ In file included from greenlet.c:5: In file included from ./greenlet.h:8: In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:27: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:33: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported #error architecture not supported ^ In file included from greenlet.c:5: In file included from ./greenlet.h:8: In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:27: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t' typedef __int64_t __darwin_blkcnt_t; /* total blocks */ ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'? typedef __int32_t __darwin_blksize_t; /* preferred block size */ ^ note: '__int128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'? typedef __int32_t __darwin_dev_t; /* dev_t */ ^ note: '__int128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'? typedef __uint32_t __darwin_gid_t; /* [???] process and group IDs */ ^ note: '__uint128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'? typedef __uint32_t __darwin_id_t; /* [XSI] pid_t, uid_t, or gid_t*/ ^ note: '__uint128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:62:9: error: unknown type name '__uint64_t' typedef __uint64_t __darwin_ino64_t; /* [???] Used for 64 bit inodes */ ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t' typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */ ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__uint128_t'? typedef __uint16_t __darwin_mode_t; /* [???] Some file attributes */ ^ note: '__uint128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:71:9: error: unknown type name '__int64_t' typedef __int64_t __darwin_off_t; /* [???] Used for file sizes */ ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'? typedef __int32_t __darwin_pid_t; /* [???] process and group IDs */ ^ note: '__int128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'? typedef __uint32_t __darwin_sigset_t; /* [???] signal set */ ^ note: '__uint128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int128_t'? typedef __int32_t __darwin_suseconds_t; /* [???] microseconds */ ^ note: '__int128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'? typedef __uint32_t __darwin_uid_t; /* [???] user IDs */ ^ note: '__uint128_t' declared here /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'? typedef __uint32_t __darwin_useconds_t; /* [???] microseconds */ ^ note: '__uint128_t' declared here In file included from greenlet.c:5: In file included from ./greenlet.h:8: In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:43:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'? typedef __uint32_t __darwin_wctype_t; ^ note: '__uint128_t' declared here In file included from greenlet.c:5: In file included from ./greenlet.h:8: In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:75: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_va_list.h:31: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/types.h:37:2: error: architecture not supported #error architecture not supported ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'xcrun' failed with exit status 1 ---------------------------------------- ERROR: Command errored out with exit status 1: /Users/arnu515/Documents/code/videochat/temp/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/62/291hqhv517j2x50bfw65mj8r0000gn/T/pip-install-kilo4pkb/greenlet/setup.py'"'"'; __file__='"'"'/private/var/folders/62/291hqhv517j2x50bfw65mj8r0000gn/T/pip-install-kilo4pkb/greenlet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/62/291hqhv517j2x50bfw65mj8r0000gn/T/pip-record-bl7qcak2/install-record.txt --single-version-externally-managed --compile --install-headers /Users/arnu515/Documents/code/videochat/temp/venv/include/site/python3.8/greenlet Check the logs for full command output. Same thing happens with gevent. Can you please help me fix this? FYI - OS: MacOS, Python: 3.8.9 [Less]

1.I have a celery task which performs web scraping [ click operations,input text on text boxes etc.] which takes 3 to 5 mins. 2.As soon as task is started i am displaying progress bar using celery progress: ... [More] https://github.com/czue/celery-progress. 3.When multiple users start executing task at the same time, the tasks are executing consecutively one after another by worker. Say 5 users started executing tasks at same time T1,T2,T3,T4,T5 .T2 starts after T1 completion, T3 starts after T2 completion, T4 starts after T3 completion, T5 starts after T4 completion etc. Please help to share the procedure to execute the celery tasks concurrently on windows. celery command used to start the worker on windows:celery -A swat worker --pool=solo -l info My application uses below configuration: ---------------------------------------- Python==3.6.2 Django==3.0.1 Celery==4.4.7 python-ldap==3.3.1 django_auth_ldap==2.2.0 mysqlclient==1.4.6 selenium curl pycurl==7.43.0.5 celery_progress==0.0.14 django_celery_results==1.2.1 setuptools==42.0.2 pywin32 requests==2.22.0 xlrd xlwt pandas openpyxl kombu==4.6.10 numpy django-crispy-forms==1.8.1 vine==1.3.0 urllib3==1.25.6 Below is my celery configuration to execute my task in background. celery.py: ========== from __future__ import absolute_import, unicode_literals import os from celery import Celery from django.conf import settings from django.db import connection # Set the default DJANGO_SETTINGS_MODULE environmental variable for the command-line program. os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'swat.settings') # You don't need this line , but it saves you from always passing in the setting module to the celery program. # This is our instance of the library where we can have many instances. app = Celery('swat') app.config_from_object('django.conf:settings', namespace='CELERY') # Load the task modules from all registered Django app configs. app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) # With the line above Celery will automatically discover tasks from all of your installed apps @app.task(bind=True) def debug_task(self): print('Request.{0!r}'.format(self.request)) print("connection schema", connection.schema_name) Please find celery settings given below. settings.py: =========== CELERY_BROKER_URL = 'amqp://guest:guest@localhost:5672//' CELERY_RESULT_BACKEND = 'django-db' CELERY_TASK_SERIALIZER = 'json' CELERY_RESULT_SERIALIZER = 'json' CELERY_TRACK_STARTED = True [Less]

Project GitHub repo: https://github.com/EdoardoCocconi/Udacity-Self-Driving-Car-Nanodegree/tree/master/Project%204%20-%20Behavioral%20Cloning I am working on a conda environment. Everything was working 1 month ago. I came back to make ... [More] some changes and when I try to import socketio or eventlet I get the error I pasted below. I haven't changed anything in the environment and tried to update flask-socketio, eventlet, and dnspython. I also created a new environment but the error persists: (carnd-term1) D:\OneDrive - The University of Nottingham\Edo\United Kingdom\Udacity\Self-Driving Car Nanodegree\Project 4 - Behavioral Cloning>python drive.py model.h5 Traceback (most recent call last): File "drive.py", line 8, in import eventlet.wsgi File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\eventlet\__init__.py", line 10, in from eventlet import convenience File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\eventlet\convenience.py", line 7, in from eventlet.green import socket File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\eventlet\green\socket.py", line 21, in from eventlet.support import greendns File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\eventlet\support\greendns.py", line 399, in resolver = ResolverProxy(hosts_resolver=HostsResolver()) File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\eventlet\support\greendns.py", line 313, in __init__ self.clear() File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\eventlet\support\greendns.py", line 316, in clear self._resolver = dns.resolver.Resolver(filename=self._filename) File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\dns\resolver.py", line 543, in __init__ self.read_registry() File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\dns\resolver.py", line 720, in read_registry self._config_win32_fromkey(key, False) File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\dns\resolver.py", line 674, in _config_win32_fromkey self._config_win32_domain(dom) File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\dns\resolver.py", line 639, in _config_win32_domain self.domain = dns.name.from_text(str(domain)) File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\dns\name.py", line 889, in from_text return from_unicode(text, origin, idna_codec) File "C:\Users\Edoardo\miniconda3\envs\carnd-term1\lib\site-packages\dns\name.py", line 852, in from_unicode raise EmptyLabel dns.name.EmptyLabel: A DNS label is empty. This is my host file: # Copyright (c) 1993-2009 Microsoft Corp. # # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. # # This file contains the mappings of IP addresses to host names. Each # entry should be kept on an individual line. The IP address should # be placed in the first column followed by the corresponding host name. # The IP address and the host name should be separated by at least one # space. # # Additionally, comments (such as these) may be inserted on individual # lines or following the machine name denoted by a '#' symbol. # # For example: # # 102.54.94.97 rhino.acme.com # source server # 38.25.63.10 x.acme.com # x client host # localhost name resolution is handled within DNS itself. # 127.0.0.1 localhost # ::1 localhost [Less]

The site is written in flask. I am trying to connect to an MQTT broker with parameters like this written in my config.py file: class DevelopConfig(Config): DEBUG = True TESTING = True ASSETS_DEBUG = True class ... [More] MqttConfig(object): MQTT_BROKER_URL = 'temper.chost.com.ua' MQTT_BROKER_PORT = 1883 MQTT_CLIENT_ID = 'flask_mqtt' MQTT_CLEAN_SESSION = True MQTT_USERNAME = 'test' MQTT_PASSWORD = 'tester' MQTT_KEEPALIVE = 2 MQTT_TLS_ENABLED = False MQTT_LAST_WILL_TOPIC = 'home/lastwill' MQTT_LAST_WILL_MESSAGE = 'bye' MQTT_LAST_WILL_QOS = 2 The connection takes a very long time or does not take place at all. The problem is most likely in the wrong organization of streams. I use the eventlet.greenthread module to create threads, and Flask-MQTT to work with the MQTT server. Flask init file init.py: from flask import Flask from flask_sqlalchemy import SQLAlchemy from flask_migrate import Migrate from flask_mail import Mail from flask_script import Manager from flask_socketio import SocketIO from flask_mqtt import Mqtt import eventlet from config import DevelopConfig, MqttConfig, MailConfig eventlet.monkey_patch() app = Flask(__name__) app.config.from_object(DevelopConfig) # применение конфигурация разработчика app.config.from_object(MqttConfig) # конфигурация для работы с сервером MQTT app.config.from_object(MailConfig) # конфигурация для работы с email db = SQLAlchemy(app) migrate = Migrate(app, db) mail = Mail(app) manager = Manager(app, db) socketio = SocketIO(app) mqtt = Mqtt(app) from app import models if __name__ == "__main__": manager.run() Launching a Flask application is done through the app.py file: # обработчик страницы профиля пользователя # только для авторизованных пользователей @app.route('/profile/') @login_required def profile(iduser): user = Users.query.get(iduser) devices = user.devices.all() print(devices) if len(devices) != 0: mqttToaa = MqttTOAA(devices[1].device_code, devices[1].typedev.reverse) return render_template('profile.html', iduser=current_user.get_id()) if __name__ == '__main__': try: socketio.run(app, host='127.0.0.1', port=5000, use_reloader=False, debug=True) except socket.error as socketerror: print("Error socketio: " + socketerror) File for working with MQTT server: # -*- coding: utf-8 -*- from app import socketio, mqtt from flask import request import eventlet import eventlet.greenthread as greenthread import json class MqttTOAA(object): # топик контроля воротами забора, топик данных воротами забора, топик контроля гаражными, топик данных гаражными type_topic = ["/Control", "/Data"] m_request_state = {"comm": "3"} # запрос на получение статуса ворот m_start = {"Gate": "Start"} # сообщение для открытия/закрытия ворот m_stop = {"Gate": "Stop"} # сообщение для остановки ворот qos_request = 2 qos_sub = 0 # состояние ворот: действие dict_state_button_fence = {"con_Clos": "Открыть", "con_Open": "Закрыть", "fl_OpenClos": ("Остановить", "Продолжить")} dict_state_button_garage = {"con_Clos": "Открыть", "con_Open": "Закрыть", "fl_OpenClos": ("Продолжить", "Остановить", "Прервать")} # действие: статус, отображаемый на странице dict_state_text_fence = {"Открыть": "закрыты", "Закрыть": "открыты", "Остановить": ("открываются", "закрываются", "в движении"), "Продолжить": "остановлены"} dict_state_text_garage = {"Открыть": "закрыты", "Закрыть": "открыты", "Продолжить": "остановлены", "Прервать": "закрываются", "Остановить": ("открываются", "закрываются", "в движении")} # поля: итоговые статусы dict_type_element_fence = {"button": "", "text": ""} # текст кнопки и отображаемый статус dict_type_element_garage = {"button": "", "text": ""} state_gate_fence = {} # статус ворот забора state_gate_garage = {} # статус ворот гаража initial_position_fence = "" # первоначальное положение position_garage = {"state": "", "stop": False} # предыдущая позиция ворот и отметка о том, были ли отсановленны POOL_TIME = 3 # Интервал отправки запроса брокеру def __init__(self, device_code, reverse): self.mqtt_connect = mqtt.on_connect()(self._handle_connect) self.mqtt_onmessage = mqtt.on_message()(self._handle_mqtt_message) self.mqtt_onlog = mqtt.on_log()(self._handle_logging) self.socketio_error = socketio.on_error()(self._handle_error) self.handle_change_state = socketio.on('change_state')(self._handle_change_state) self.device_code = "BK" + device_code self.reverse = reverse # обработчик ошибок def _handle_error(self): print(request.event["message"]) # "my error event" print(request.event["args"]) # (data,) # функция изменения состояния ворот по нажатию def _handle_change_state(self): message = None try: # для каких из ворот необходимо сменить состояние if self.reverse is not True: type_g = self.state_gate_fence else: type_g = self.state_gate_garage if type_g["fl_OpenClos"] == 1: # ворота в движении -> остановка message = self.m_stop # остановились и двигаются в обратном направлении -> остановка elif (type_g["fl_OpenClos"] == 0) and (self.position_garage["state"] == "закрываются"): message = self.m_stop self.position_garage["state"] = "открываются" else: # ворота остановленны -> продолжение движения message = self.m_start print(self.position_garage["state"]) print(message) except Exception as ex: print(ex) mqtt.publish(self.device_code + self.type_topic[0], json.dumps(message), self.qos_request) # передача запроса на получение данных @staticmethod def handle_publish(topic_req_res, m_req_state, qos_req, timer): while True: print("Send") eventlet.sleep(timer) msg = json.dumps(m_req_state) mqtt.publish(topic_req_res, msg, qos_req) # ожидание подключения к брокеру, # затем подписка на топик и запуск потока для постоянной отсылки сообщений в топик Control def _handle_connect(self, client, userdata, flags, rc): mqtt.subscribe(self.device_code + self.type_topic[1], self.qos_sub) print("Subscribe!") publish_thread = greenthread.spawn(self.handle_publish, self.device_code + self.type_topic[0], self.m_request_state, self.qos_request, self.POOL_TIME) # обработка принятых сообщений от топика, на который подписан def _handle_mqtt_message(self, client, userdata, message): print("Get message") data = dict( topic=message.topic, payload=message.payload.decode(), qos=message.qos, ) try: data = json.loads(data['payload']) if self.reverse is not True: self.fence_msg(data) else: self.garage_msg(data) except Exception as ex: print("Exception: " + str(ex)) 2 messages are constantly displayed in the logs: 16 Sending PINGREQ 16 Received PINGRESP The code hangs waiting for the decorator to be called: mqtt.on_connect() [Less]

Tried everything on the internet but nothing worked. I Have two flask applications that both running on SocketIO: App 1 running on port 5000. App 2 running on port 5001 that creates thread on port 6002 Our Goal in to send data from first ... [More] app to the second app. When App 1 sends data to App 2 that what happens: Creates a new pipe to App2 def create_pipe(data): address = ('localhost', 6002) try: conn = Client(address, authkey=b'abc') conn.send(data) conn.close() except Exception as e: print(e) return App 2 is listening on port 6002 and should receive the data with those functions: @socketio.on('connect') def start_chats_thread(): """ Creates new background task that starts a thread to listen for incoming data """ global thread with thread_lock: if thread is None: thread = socketio.start_background_task(new_messages_job) def new_messages_job(): """ Should Get Data from App 1 and call to send_message() that send data to users """ address = ('localhost', 6002) listener = Listener(address, authkey=b'abc') while True: conn = listener.accept() msg = conn.recv() send_message(msg[0], users_ids=msg[1]) listener.close() App 2 doesn't get data from App 1 with the following error: [WinError 10035] A non-blocking socket operation could not be completed immediately And after that error the thread shuts down. We think that there's a problem in listener.accept(). Help will be much appreciated. [Less]

I am running a Flask production server using flask-socketio and eventlet and when trying to submit a form which contains a file to upload, Flask fails to read the entire request. This occurs when the file reaches above a few kb (around ... [More] 50kb or more). The file I am trying to upload is a 60kb .txt file with a word on each line. Things work as expected with smaller file sizes of 1-2kb. def get_file(request): if 'uploadFile' not in request.files: return redirect(request.url) return = request.file['uploadFile'] Having done some tests I was able to determine that the code does not get passed checking request.files. As an example, I would try to print(request.files) and the code would not move on from there, it would just hang. I understand that Flask's built-in methods may not be the most effecient so I found a library streaming-form-data that can assist with loading large files and so implemented this as a replacement def get_file(request): parser = StreamingFormDataParser(headers=request.headers) parser.register('file', FileTarget('/temp/file.txt') while True: chunk = request.stream.read(8192) if not chunk: break parser.data_received(chunk) # add read bytes to file socketio.sleep(1) The file would still not finish uploading however I was able to find that it stops when trying to do chunk = request.stream.read(8192). This wouldn't happen straight away, it would usually stop around 4-5th iteration. Tested with and without socketio.sleep() as I understand this can be needed with tasks that require more time. I did some more testing using the Flask Dev Server and found that if I use app.run(threaded=True) it would complete as expected and the code would continue. However I haven't been able to get it to work using socketio.run() and eventlet, example of my main.py import eventlet eventlet.monkey_patch() from app import app, socketio if __name__ == "__main__": # app.run(threaded=True) # This will work and finish loading the file socketio.run(app) # does not work During the while loop to read the stream I do not receive any errors, it merely does not continue to to get the next chunk = request.stream.read(8192) [Less]

Im hosting a chat app that uses flask-Socketio, flask, Gunicorn, Eventlet. I have tried following deployment instructions from Flask-Sockeio Documentation but I get Broken Pipe Error: Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 ... [More] gunicorn[31116]: self._send_bytes(m[offset:offset + size]) Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 gunicorn[31116]: File "/usr/lib/python3.6/multiprocessing/connection.py", line 404, in _send_bytes Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 gunicorn[31116]: self._send(header + buf) Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 gunicorn[31116]: File "/usr/lib/python3.6/multiprocessing/connection.py", line 368, in _send Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 gunicorn[31116]: n = write(self._handle, buf) Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 gunicorn[31116]: File "/var/www/web_chatbot/venv/lib/python3.6/site-packages/eventlet/green/os.py", line 64, in write Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 gunicorn[31116]: return __original_write__(fd, st) Aug 24 12:18:20 ubuntu-s-4vcpu-8gb-fra1-01 gunicorn[31116]: BrokenPipeError: [Errno 32] Broken pipe my Pip Environment Requirements is absl-py==0.9.0 astor==0.8.1 astunparse==1.6.3 cachetools==4.1.1 certifi==2020.6.20 chardet==3.0.4 click==7.1.2 dataclasses==0.7 dnspython==1.16.0 eventlet==0.26.1 filelock==3.0.12 Flask==1.1.2 Flask-Cors==3.0.8 Flask-SocketIO==4.3.1 gast==0.3.3 google-auth==1.20.1 google-auth-oauthlib==0.4.1 google-pasta==0.2.0 greenlet==0.4.16 grpcio==1.31.0 gunicorn==20.0.4 h5py==2.10.0 idna==2.10 importlib-metadata==1.7.0 itsdangerous==1.1.0 Jinja2==2.11.2 joblib==0.16.0 Keras-Applications==1.0.8 Keras-Preprocessing==1.1.2 Markdown==3.2.2 MarkupSafe==1.1.1 monotonic==1.5 numpy==1.18.5 oauthlib==3.1.0 opt-einsum==3.3.0 packaging==20.4 protobuf==3.13.0 pyasn1==0.4.8 pyasn1-modules==0.2.8 pyparsing==2.4.7 python-engineio==3.13.1 python-socketio==4.6.0 regex==2020.7.14 requests==2.24.0 requests-oauthlib==1.3.0 rsa==4.6 sacremoses==0.0.43 scipy==1.4.1 sentencepiece==0.1.91 six==1.15.0 tb-nightly==1.14.0a20190603 tensorboard==2.3.0 tensorboard-plugin-wit==1.7.0 tensorflow==2.3.0 tensorflow-estimator==2.3.0 termcolor==1.1.0 tf-estimator-nightly==1.14.0.dev2019060501 tokenizers==0.8.1rc1 tqdm==4.48.2 transformers==3.0.2 urllib3==1.25.10 Werkzeug==1.0.1 wrapt==1.12.1 zipp==3.1.0 Also this can help i have other error but I think they do not relate, on this link Connection Refused Error nginx Flask_socketio, Flask App On the link you can find my NGINX config file. [Less]

So I have been setting up a website with flask + flask_socketio + eventlet + gunicorn + nginx. I have looked at the documentation for flask_socketio for this configuration and it is supported. Though after using the website I noticed ... [More] that while flask would work perfectly, flask_socketio would not work at all. Here are the relevant configuration and error codes. I would like to note that some flask_socketio bits post requests would go through with a 200 OK. This is on ubuntu 18.04 Nginx: user www-data; worker_processes auto; pid /run/nginx.pid; include /etc/nginx/modules-enabled/*.conf; events { worker_connections 65535; multi_accept on; } http { ## # Basic Settings ## charset utf-8; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; server_tokens off; log_not_found off; client_max_body_size 16M; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; ## # SSL Settings ## ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE ssl_prefer_server_ciphers on; ## # Logging Settings ## access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## # Gzip Settings ## gzip on; # gzip_vary on; # gzip_proxied any; # gzip_comp_level 6; # gzip_buffers 16 8k; # gzip_http_version 1.1; # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; ## # Virtual Host Configs ## server { listen 80; server_name http://compoundingly.com; location / { proxy_pass http://0.0.0.0:5000; proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } location /static { alias /home/shorya/compoundingly_website/compoundingly_code/static; expires 30d; } location /socket.io { include proxy_params; proxy_http_version 1.1; proxy_buffering off; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "Upgrade"; proxy_pass http://0.0.0.0:5000/socket.io; } } include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; } Gunicorn service [Unit] Description=Gunicorn instance to serve myproject Requires=gunicorn.socket After=network.target [Service] User=root Group=www-data WorkingDirectory=/home/shorya/compoundingly_website/compoundingly_code Environment="PATH=/home/shorya/compoundingly_website/compoundingly_code/compoundingly_venv/bin" ExecStart=/home/shorya/compoundingly_website/compoundingly_code/compoundingly_venv/bin/gunicorn -k eventlet --workers 3 --bind unix:compoundingly_code.sock -m 007 wsgi:app Restart=on-failure [Install] WantedBy=multi-user.target wsgi.py from app import app if __name__ == "__main__": app.run() pip3 freeze click==7.1.2 dnspython==1.16.0 eventlet==0.26.1 Flask==1.1.2 Flask-SocketIO==4.3.1 greenlet==0.4.16 gunicorn==19.7.1 itsdangerous==1.1.0 Jinja2==2.11.2 MarkupSafe==1.1.1 monotonic==1.5 pkg-resources==0.0.0 python-engineio==3.13.1 python-socketio==4.6.0 six==1.15.0 Werkzeug==1.0.1 failed request URL: http://compoundingly.com/socket.io/?EIO=3&transport=polling&t=NFGwVcP&sid=22ad9ddeaffe4eeb8525927aca86806b Status: 400 BAD REQUEST Source: Network Address: 192.241.149.99:80 Initiator: index.js:83 good request URL: http://compoundingly.com/socket.io/?EIO=3&transport=polling&t=NFGwVtd Status: 200 OK Source: Network Address: 192.241.149.99:80 Initiator: index.js:83 502 bad gateway request URL: http://compoundingly.com/socket.io/?EIO=3&transport=polling&t=NFGwdji&sid=69f16b45a1aa474294cdeb2452f0e71a Status: 502 Bad Gateway Source: Network Address: 192.241.149.99:80 Initiator: index.js:83 Gunicorn error log Aug 08 22:09:16 sm01 gunicorn[1956]: [2020-08-08 22:09:16 -0400] [5964] [INFO] Booting worker with pid: 5964 Aug 08 22:09:47 sm01 gunicorn[1956]: [2020-08-08 22:09:47 -0400] [1956] [CRITICAL] WORKER TIMEOUT (pid:5962) Aug 08 22:09:47 sm01 gunicorn[1956]: [2020-08-08 22:09:47 -0400] [5962] [INFO] Worker exiting (pid: 5962) Aug 08 22:09:47 sm01 gunicorn[1956]: [2020-08-08 22:09:47 -0400] [5977] [INFO] Booting worker with pid: 5977 Aug 08 22:09:48 sm01 gunicorn[1956]: [2020-08-08 22:09:48 -0400] [1956] [CRITICAL] WORKER TIMEOUT (pid:5963) Aug 08 22:09:48 sm01 gunicorn[1956]: [2020-08-08 22:09:48 -0400] [1956] [CRITICAL] WORKER TIMEOUT (pid:5964) Aug 08 22:09:48 sm01 gunicorn[1956]: [2020-08-08 22:09:48 -0400] [5963] [INFO] Worker exiting (pid: 5963) Aug 08 22:09:48 sm01 gunicorn[1956]: [2020-08-08 22:09:48 -0400] [5964] [INFO] Worker exiting (pid: 5964) Aug 08 22:09:48 sm01 gunicorn[1956]: [2020-08-08 22:09:48 -0400] [5981] [INFO] Booting worker with pid: 5981 Aug 08 22:09:48 sm01 gunicorn[1956]: [2020-08-08 22:09:48 -0400] [5982] [INFO] Booting worker with pid: 5982 For people who are facing this same problem there are a few things you can do to make the system work after you have done this: I changed my wsgi to: from app import * if __name__ == "__main__": #app.run() app = Flask(__name__)#,template_folder="./dist/app/templates" app.config['SECRET_KEY'] = 'secret!' socketio = SocketIO(app,async_mode="eventlet") socketio.run(app,host="0.0.0.0",port=5000,debug=True) added --bind 0.0.0.0:5000 to the gunicorn command. I also downloaded a newer version of gunicorn but not the newest. 19.7 does not work 20.0.3 works 20.0.4 does not work [Less]