25 Sidekiq processes for Gitlab

Question

Looking at htop output on my server I see 25 sidekiq processes spawned by Gitlab. I use Gitlab privately, so there is never going to be any load, so I doubt all of those processes are required, but I cannot see how to configure their number.

Is there actually any point for me to bother about that on a resource-restricted server?

No, I'm afraid I left it be the way it is. Mostly because I couldn't figure out how to set the sidekiq config for gitlab like @md_5 suggested in his answer. — Alexander Filatov, Jul 17 '14 at 07:25
I solved my memory issue (Gitlab & Node.js) by increasing the swap, but I still have the 25 Sidekiq processes there, so if there is any advances, I'd be happy to know. — Alejandro Piad, Jul 17 '14 at 22:25

score 3 · Answer 1 · answered Mar 25 '14 at 20:31

3

Sure, check this thread here: https://github.com/gitlabhq/gitlabhq/issues/2780

Just edit the sidekiq config.yml, note the concurrency option: https://github.com/mperham/sidekiq/blob/master/examples/config.yml

answered Mar 25 '14 at 20:31

md_5

376
2
11

1

Hi, I'm also using Gitlab, and as of version 6.7.3 there is no config file that I know of specifically for Sidekiq. Is there a workaround for this? – Alejandro Piad Jul 16 '14 at 23:06

score 3 · Answer 2 · answered Aug 05 '14 at 02:45

I edited the Sidekiq startup arguments. In GitLab <7.0.0 it's under scripts/background_jobs but in >7.0.0 it's under bin/background_jobs

Change:

function start_sidekiq
{
  bundle exec sidekiq -q post_receive -q mailer -q system_hook -q project_web_hook -q gitlab_shell -q common -q default -e $RAILS_ENV -P $sidekiq_pidfile $@ >> $sidekiq_logfile 2>&1
}

To:

function start_sidekiq
{
  bundle exec sidekiq -c 10 -q post_receive -q mailer -q system_hook -q project_web_hook -q gitlab_shell -q common -q default -e $RAILS_ENV -P $sidekiq_pidfile $@ >> $sidekiq_logfile 2>&1
}

Notice the -c 10. Change that to whatever you want.

score 2 · Answer 3 · answered Jul 26 '17 at 18:55

2

In Debian install of version 9.3.0 I had /etc/gitlab/gitlab.rb which have configuration lines for sidekiq.

Change

# sidekiq['concurrency'] = 25

to whatever number you seem fit:

sidekiq['concurrency'] = 5

(Reason I changed my self was because the default 25 processes ate a lot of ram causing swap to be used, in turn making gitlab real slow. Performance improved alot for me after this change)

answered Jul 26 '17 at 18:55

Brimstedt

151
1
12

Yep. This is the way to go! Don't forget to run: `gitlab-ctl reconfigure`. Any reason why you selected 5? – Melroy van den Berg Mar 28 '18 at 17:12

score 1 · Answer 4 · answered Sep 10 '17 at 07:51

Most of the proposed solutions to this problem both in this Q&A thread and elsewhere on line seem to be out of date, but the problem is still current, so here is my solution for Gitlab 9.5.3 on Archlinux using the community packages:

I was unable to get this to work by adding a sidekick.yml, sidekick_queues.yml, or to anything else in /etc and resorted to hacking the installed package source directly.

Edit the system file /usr/share/webapps/gitlab/config/sidekiq_queues.yml and add this line just after the opening --- YAML marker:

:concurrency: 5

The resulting YAML looks something like this:

selection_414

Then sudo systemctl restart gitlab-sidekiq and I finally got only 5 threads chewing through memory instead of 25.

score 0 · Answer 5 · answered Aug 30 '16 at 12:27

For me it worked to just go to /home/git/gitlab/config. There was a sidekiq.yml.example file. I just ran:

$ cd /home/git/gitlab/config
$ cp sidekiq.yml.example sidekiq.yml

Using vim sidekiq.yml you will see that there is a :concurrency: option. Set it to the number of sidekiq processes you wish, save the file, and run service gitlab restart.

Disclaimer: Location of your GitLab installation folder may vary. For me it was /home/git/gitlab

score 0 · Answer 6 · answered Sep 08 '17 at 09:41

I have a "from source" installed gitlab version and I had to edit config/sidekiq_queues.yml and add :concurrency: X (where X is your desired number of processes.

The sidekiq.yml is not used by gitlab. You can see that if you look at the running process and its -C option.

25 Sidekiq processes for Gitlab

6 Answers6

Linked