Skip to content

Conversation

@nishidayuya
Copy link

Sometimes, ActiveRecord::Deadlocked is occurred in Delayed::Backend::ActiveRecord::Job.clear_locks!. And some jobs are still locked.

F, [2019-06-20T11:30:53.459511 #123099] FATAL -- : Mysql2::Error: Deadlock found when trying to get lock; try restarting transaction: UPDATE `delayed_jobs` SET `delayed_jobs`.`locked_by` = NULL, `delayed_jobs`.`locked_at` = NULL WHERE `delayed_jobs`.`locked_by` = 'delayed_job.1 host:XXXXX pid:123099 (ActiveRecord::Deadlocked)

worker_name is generated with PID. So, Delayed::Backend::ActiveRecord::Job.clear_locks! is retryable method. This pull-request makes Delayed::Backend::ActiveRecord::Job.clear_locks! to retry when ActiveRecord::Deadlocked is occurred.

Before pull-request

  • When ActiveRecord::Deadlocked is occurred in Delayed::Backend::ActiveRecord::Job.clear_locks!, some jobs are still locked.

After pull-request

  • When ActiveRecord::Deadlocked is occurred in Delayed::Backend::ActiveRecord::Job.clear_locks!, retry it (max 10-times) and clear locked jobs.

@nishidayuya nishidayuya force-pushed the retryable-clear_locks branch from 28f70dc to 18affa0 Compare August 19, 2019 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant