Replies: 1 comment
-
|
Hello Airflow community, I'm currently facing the same issue as mentioned above. I have configured airflow 3.0.6, celery executor, postgressql DB, using FAB OAuth. Currently testing it out on a VM. Everything was running fine but the VM got rebooted in the weekend and now when I start the api-server and try to access the UI, it gives be the same error as mentioned above by @ahipp13 ((psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"). |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi Airflow Community,
My co-worker and I have been struggling all week to figure out a problem we are having with our airflow instance and are looking anywhere for help.
For background, we are running airflow 2.4.3 on Kubernetes using the 8.6.1 Airflow Community Helm Chart (https://github.com/airflow-helm/charts/tree/main/charts/airflow) and we are using a postgreSQL DB running on an external server as our database.
To start, we were previously running Airflow 2.2.5 and everything was running smoothly, including the OAuth we had implemented. We then upgraded our helm chart and airflow version to 2.4.3, and our OAuth implementation broke. When we try to load the webserver login page we get session id errors in the logs of the webserver pod (Those logs are down below). Along with that, it will load the page, but it will be a different weird style and look with every refresh:
When you hit the login button, these logs pop up:
[2022-11-23 15:58:07,072] {views.py:659} ERROR - Error authorizing OAuth access token: mismatching_state: CSRF Warning! State not equal in request and response. ││ airflow-web [2022-11-23 15:58:07,072] {views.py:659} ERROR - Error authorizing OAuth access token: mismatching_state: CSRF Warning! State not equal in request and response.
And then it brings you back to the login screen where you again get the same errors I talked about before. I do not know if these errors are related or not. Any interesting note here is if we turn off OAuth and go to the default webserver_config, everything works just fine. Our implementation of the webserver_config is down below.
If anybody could help give answers as to what you think is going on or give tips on how to debug that would be greatly appreciated. This is my first time posting so if any other additional info is needed please let me know. Thank you
Webserver_config
Webserver logs
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /login/ HTTP/1.1" 200 16330 "https://login.microsoftonline.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/font-awesome.min.css HTTP/1.1" 200 0 "https://dev-edw-airflow.hcck8s-ctc-np1.optum.com/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web [2022-11-23 15:35:59,423] {app.py:1741} ERROR - Exception on /static/appbuilder/css/bootstrap.min.css [GET]
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web return super().save_session(*args, **kwargs)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 578, in save_session
airflow-web self.db.session.commit()
airflow-web File "", line 2, in commit
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1428, in commit
airflow-web self._transaction.commit(_to_root=self.future)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829, in commit
airflow-web self._prepare_impl()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
airflow-web self.session.flush()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3345, in flush
airflow-web self._flush(objects)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3485, in flush
airflow-web transaction.rollback(capture_exception=True)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
airflow-web compat.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3445, in _flush
airflow-web flush_context.execute()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
airflow-web rec.execute(self)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
airflow-web util.preloaded.orm_persistence.save_obj(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 244, in save_obj
airflow-web _emit_insert_statements(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1221, in _emit_insert_statements
airflow-web result = connection._execute_20(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20
airflow-web return meth(self, args_10style, kwargs_10style, execution_options)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
airflow-web return connection._execute_clauseelement(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
airflow-web ret = self._execute_context(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
airflow-web self.handle_dbapi_exception(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception
airflow-web util.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd53ad71b0>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj)
airflow-web
airflow-web During handling of the above exception, another exception occurred:
airflow-web
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1844, in finalize_request
airflow-web response = self.process_response(response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2340, in process_response
airflow-web self.session_interface.save_session(self, ctx.session, response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/session.py", line 33, in save_session
airflow-web return super().save_session(*args, **kwargs)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 553, in save_session
airflow-web saved_session = self.sql_session_model.query.filter_by(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2810, in first
airflow-web return self.limit(1)._iter().first()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2894, in _iter
airflow-web result = self.session.execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1688, in execute
airflow-web conn = self._connection_for_bind(bind)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1529, in _connection_for_bind
airflow-web return self._transaction._connection_for_bind(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 721, in _connection_for_bind
airflow-web self._assert_active()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 601, in _assert_active
airflow-web raise sa_exc.PendingRollbackError(
airflow-web sqlalchemy.exc.PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd53ad71b0>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj) (Background on this error at: https://sqlalche.me/e/14/7s2a)
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/flags/flags16.css HTTP/1.1" 500 1575 "https://dev-edw-airflow.hcck8s-ctc-np1.optum.com/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/ab.css HTTP/1.1" 500 1575 "https://dev-edw-airflow.hcck8s-ctc-np1.optum.com/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/dist/airflowDefaultTheme.731e57571b52cca4350d.css HTTP/1.1" 200 0 "https://dev-edw-airflow.hcck8s-ctc-np1.optum.com/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web [2022-11-23 15:35:59,534] {app.py:1741} ERROR - Exception on /static/dist/materialIcons.f9559e4953177b8b9a4a.css [GET]
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web
airflow-web The above exception was the direct cause of the following exception:
airflow-web
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
airflow-web response = self.full_dispatch_request()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request
airflow-web return self.finalize_request(rv)
airflow-web return meth(self, args_10style, kwargs_10style, execution_options)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
airflow-web return connection._execute_clauseelement(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
airflow-web ret = self._execute_context(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
airflow-web self.handle_dbapi_exception(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception
airflow-web util.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd4c9fd900>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 574672, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj)
airflow-web [2022-11-23 15:35:59,591] {app.py:1741} ERROR - Exception on /static/dist/loadingDots.5da42d00b5455806e709.css [GET]
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web
airflow-web The above exception was the direct cause of the following exception:
airflow-web
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
airflow-web response = self.full_dispatch_request()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request
airflow-web return self.finalize_request(rv)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1844, in finalize_request
airflow-web response = self.process_response(response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2340, in process_response
airflow-web self.session_interface.save_session(self, ctx.session, response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/session.py", line 33, in save_session
airflow-web return super().save_session(*args, **kwargs)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 578, in save_session
airflow-web self.db.session.commit()
airflow-web File "", line 2, in commit
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1428, in commit
airflow-web self._transaction.commit(_to_root=self.future)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829, in commit
airflow-web self._prepare_impl()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
airflow-web self.session.flush()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3345, in flush
airflow-web self._flush(objects)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3485, in flush
airflow-web transaction.rollback(capture_exception=True)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
airflow-web compat.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3445, in _flush
airflow-web flush_context.execute()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
airflow-web rec.execute(self)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
airflow-web util.preloaded.orm_persistence.save_obj(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 244, in save_obj
airflow-web _emit_insert_statements(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1221, in _emit_insert_statements
airflow-web result = connection._execute_20(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20
airflow-web return meth(self, args_10style, kwargs_10style, execution_options)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
airflow-web return connection._execute_clauseelement(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
airflow-web ret = self._execute_context(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
airflow-web self.handle_dbapi_exception(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception
airflow-web util.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
Beta Was this translation helpful? Give feedback.
All reactions