using portage eix for the first time – cannot open database file

Installing “app-portage/eix” in Gentoo to manage your portage updates you might encounter this error, when trying to use “eix” for the first time:

Writing database file /var/cache/eix/portage.eix...
cannot open database file /var/cache/eix/portage.eix for writing (mode = 'wb')

The chances are missing directory “/var/cache/eix/” or the user:group of the “/var/cache/eix/” is root:root, which is NOT right.

The user:group must be “portage:portage”.

So the solution is really simple:

mkdir -p /var/cache/eix
chown portage:portage /var/cache/eix

Output – the errors you might get

Using the eix-sync failed with:

root@srv1 ~ # eix-sync 
 * eix-cache does not exist
 * Running eix-update
Reading Portage settings...
Building database (/var/cache/eix/portage.eix)...
[0] "gentoo" /usr/portage/ (cache: metadata-md5-or-flat)
     Reading category 167|167 (100) Finished             
[1] "myportage" /usr/local/myportage (cache: parse|ebuild*#metadata-md5#metadata-flat#assign)
     Reading category 167|167 (100) Finished    
Applying masks...
Calculating hash tables...
Writing database file /var/cache/eix/portage.eix...
cannot open database file /var/cache/eix/portage.eix for writing (mode = 'wb')
 * eix-update failed
 * Time statistics:
     6 seconds for initial eix-update
     6 seconds total

Using the “eix-update” failed, too.

root@srv ~ # eix-update 
Reading Portage settings...
Building database (/var/cache/eix/portage.eix)...
[0] "gentoo" /usr/portage/ (cache: metadata-md5-or-flat)
     Reading category 167|167 (100) Finished             
[1] "myportage" /usr/local/myportage (cache: parse|ebuild*#metadata-md5#metadata-flat#assign)
     Reading category 167|167 (100) Finished    
Applying masks...
Calculating hash tables...
Writing database file /var/cache/eix/portage.eix...
cannot open database file /var/cache/eix/portage.eix for writing (mode = 'wb')

Output 2 – Successful update with eix

root@srv ~ # eix-update 
Reading Portage settings...
Building database (/var/cache/eix/portage.eix)...
[0] "gentoo" /usr/portage/ (cache: metadata-md5-or-flat)
     Reading category 167|167 (100) Finished             
[1] "myportage" /usr/local/myportage (cache: parse|ebuild*#metadata-md5#metadata-flat#assign)
     Reading category 167|167 (100) Finished    
Applying masks...
Calculating hash tables...
Writing database file /var/cache/eix/portage.eix...
Database contains 19544 packages in 167 categories

Kernel loads only single processor on multi-processor system – ACPI: x2apic entry ignored

There multiple reports on this issue with different processors

kernel, which worked perfectly on multiple systems, loads on our new server and only one processor is shown

Just for the record, the SMP is enabled in the kernel (and in the BIOS – Hyperthreading and multicores are enabled, too):

root@srv ~ # zcat /proc/config.gz | grep 'CONFIG_SMP'
CONFIG_SMP=y

The problem is x2APIC Support in the BIOS of your server.

Apparently, our kernel (version 4.18.12) missed the kernel feature:

root@srv ~ # zcat /proc/config.gz | grep -i 'x2apic'
root@srv

You can see no kernel configuration entry “CONFIG_X86_X2APIC=y” is shown from the above command.

And if your BIOS enables the support of x2APIC you may end up with just one processor under Linux.

This was the case in our server. The x2APIC support is enabled in the BIOS and our kernel (version 4.18.12) does not have CONFIG_X86_X2APIC enabled.
To fix this issue you first might disable the feature in the BIOS and you are going to have all your processors shown and they could be used to compile fast your new kernel (of course, in the case you use custom kernel) after you enable the feature in the kernel CONFIG_X86_X2APIC, which is under
Kernel Configuration —> [*] Support x2apic. The asterisk means it is enabled, so build your kernel. Check and enable this “Device Drivers –> IOMMU Hardware Support –> Support for Interrupt Remapping”, too.
Here you can see how to enable and disable processor x2APIC support in HP ProLiant DL160 Gen9 (2 processors Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz) – Enable or Disable the processor x2APIC support in HP ProLiant DL160 Gen9.
Keep on reading!

bind – dns server queries statistics with statistics-file

There is an option “statistics-file” in the BIND9 configuration for query statistics. It will give you statistics for

  • Incoming Requests – total number of queries
  • Incoming Queries – queries by record type
  • Outgoing Queries – queries by record type per view
  • Name Server Statistics – extended queries statistics by network connection type (UDP, TCP or IPv4 and IPv6 interface), by type of answer (authoritative, non authoritative and so on) and more
  • Zone Maintenance Statistics – transfer and system queries
  • Resolver Statistics – recursive queries
  • Cache DB RRsets – cached resources records sets
  • Socket I/O Statistics – statistics numbers for UDP, TCP (for both IPv4 and IPV6) sockets and connections opened, closed, failure
  • Per Zone Query Statistics – so you can see how many queries you have for a zone in a view (and the transfers if the server is a slave)

In named.conf:

options {
....
    statistics-file "/var/log/named.stats";
    zone-statistics yes ;
....

But if you check in /var/log this file might be missing even your BIND server has been running for months!

This is because the statistics and the file is generated on request and is a snapshot at the moment you do the request

To request from the BIND server to generate the file is pretty easy:

root@srv ~ # rndc stats
root@srv ~ #

No standard output and you should have the stats file generated:

root@srv ~ # ls -altr /var/log/named.stats 
-rw-r--r-- 1 named named 174997 Apr  7 01:43 /var/log/named.stats

The generated requests are appended in the file with a UNIX timestamp.

....
--- Statistics Dump --- (1550561292)
+++ Statistics Dump +++ (1551233218)
....

Keep on reading!

Gentoo kde-frameworks/kdewebkit failed compilation with Qt5WebKit could not be found because dependency is required

Updating the KDE Plasma Desktop in our Gentoo workstations this time failed with

CMake Error at /usr/share/cmake/Modules/CMakeFindDependencyMacro.cmake:48 (find_package):
  Found package configuration file:

    /usr/lib64/cmake/Qt5WebKit/Qt5WebKitConfig.cmake

  but it set Qt5WebKit_FOUND to FALSE so package "Qt5WebKit" is considered to
  be NOT FOUND.  Reason given by package:

  Qt5WebKit could not be found because dependency is required to have exact
  version 5.11.x.

It was strange because the previous emerge included the QT upgrade from old 5.11.2 to 5.12.1 and this dependency should have been resolved properly before:

emerge -vau $(qlist -IC|grep dev-qt|sort|uniq)

But apparently despite that the emerge built all QT libraries in dependency order the “dev-qt/qtwebkit” was built against the old QT libraries. And this is what is saying the above error!

The solution is really simple just rebuild the dev-qt/qtwebkit

root@srv ~ # emerge -va dev-qt/qtwebkit

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-qt/qtwebkit-5.212.0_pre20180120:5/5.212::gentoo  USE="X geolocation hyphen jit multimedia opengl printsupport qml -gles2 -gstreamer -nsplugin 
-orientation -webp" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] yes

Keep on reading!

Installing and running BOINC client (with SETI project) under CentOS 7

Here is how you can install a BOINC client and attach it to a project (SETI). We use only command line tools, not GUI involved here. You can attach to a project and do some various administrative work with

boinccmd

like to get the progress of the currently running project tasks, project info, manage tasks and more.
Steps to install and run a (SETI) project:

STEP 1) Install BOINC client

The installation of the BOINC client requires using EPEL repository. Become root user and install Epel repository and Boinc client.

sudo su
yum update -y
yum install -y epel-release
yum install -y boinc-client
systemctl start boinc-client
systemctl enable boinc-client

STEP 2) Attach to a project

Here we use our SETI project to attach the server to it. There are two ways to attach to a project:

  • Using URL project and account key (strong or weak – it works with both). You can get your account keys from the site project url.
  • Using URL project and account info – username and password

To attach to a project your boinc client must be up and running and to use boinccmd – The command line interface to the BOINC client:

cd /var/lib/boinc
boinccmd --project_attach "http://setiathome.berkeley.edu/" "111111_22222233333344444444444555555555"

The first argument of “–project_attach” is the URL address of the project site, in this case, SETI with “http://setiathome.berkeley.edu/” and the account key of our account is 111111_22222233333344444444444555555555 (this is not the real one!). After successful attach your client will start to download the project files and begin to work on units:

[myuser@compute1 ~]# boinccmd --get_tasks

======== Tasks ========
1) -----------
   name: blc36_2bit_guppi_58406_31023_HIP20352_0115.19579.818.22.45.146.vlar_0
   WU name: blc36_2bit_guppi_58406_31023_HIP20352_0115.19579.818.22.45.146.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:38:07 2019
   report deadline: Sat May 25 18:37:49 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4843.738587
   CPU time at last checkpoint: 1326.038000
   current CPU time: 1366.303000
   fraction done: 0.170349
   swap size: 54 MB
   working set size: 52 MB
2) -----------
   name: blc34_2bit_guppi_58406_26949_HIP20491_0103.21476.0.21.44.1.vlar_1
   WU name: blc34_2bit_guppi_58406_26949_HIP20491_0103.21476.0.21.44.1.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: uninitialized
   active_task_state: UNINITIALIZED
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 5838.283108
3) -----------
 ....

You can see all the tasks – the one running at the moment and those on the queue.
When the “fraction done” reaches 1.0 the tasks is ready to report.

To view all running work units and some useful links in the project site like forums, account, preferences, recent tasks, list of the computers on which you are running SETI@Home, team information and more you can use “–get_simple_gui_info” (some of the data here are changed):

[myuser@compute1 ~]# boinccmd --get_simple_gui_info
======== Projects ========
1) -----------
   name: SETI@home
   master URL: http://setiathome.berkeley.edu/
   user_name: neoX
   team_name: neoX Group
   resource share: 150.000000
   user_total_credit: 14376490.970263
   user_expavg_credit: 25085.116699
   host_total_credit: 0.000000
   host_expavg_credit: 0.000000
   nrpc_failures: 2
   master_fetch_failures: 0
   master fetch pending: no
   scheduler RPC pending: no
   trickle upload pending: no
   attached via Account Manager: no
   ended: no
   suspended via GUI: no
   don't request more work: no
   disk usage: 0.000000
   last RPC: Tue Apr  2 17:19:56 2019

   project files downloaded: 1554212310.422403
GUI URL:
   name: Message boards
   description: Correspond with other users on the SETI@home message boards
   URL: http://setiathome.berkeley.edu/forum_index.php
GUI URL:
   name: Help
   description: Ask questions and report problems
   URL: http://setiathome.berkeley.edu/forum_help_desk.php
GUI URL:
   name: Account
   description: View your account information
   URL: http://setiathome.berkeley.edu/home.php
GUI URL:
   name: Preferences
   description: View and modify your computing preferences
   URL: http://setiathome.berkeley.edu/prefs.php?subset=global
GUI URL:
   name: Tasks
   description: View your recent tasks
   URL: http://setiathome.berkeley.edu/results.php?userid=111111
GUI URL:
   name: Computers
   description: View a list of the computers on which you are running SETI@Home
   URL: http://setiathome.berkeley.edu/hosts_user.php?userid=111111
GUI URL:
   name: Team
   description: View information about your team: neoX Group
   URL: http://setiathome.berkeley.edu/team_display.php?teamid=22222
GUI URL:
   name: Donate
   description: Donate to SETI@home
   URL: http://setiathome.berkeley.edu/sah_donate.php
   jobs succeeded: 16
   jobs failed: 0
   elapsed time: 107328.542101
   cross-project ID: 33333333333333333333333333333333

======== Tasks ========
1) -----------
   name: blc34_2bit_guppi_58406_27281_HIP20917_0104.20977.818.21.44.184.vlar_1
   WU name: blc34_2bit_guppi_58406_27281_HIP20917_0104.20977.818.21.44.184.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4658.869936
   CPU time at last checkpoint: 1902.641000
   current CPU time: 1949.207000
   fraction done: 0.202014
   swap size: 58 MB
   working set size: 56 MB
2) -----------
   name: blc34_2bit_guppi_58406_28625_HIP21029_0108.20913.818.21.44.250.vlar_1
   WU name: blc34_2bit_guppi_58406_28625_HIP21029_0108.20913.818.21.44.250.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4586.517845
   CPU time at last checkpoint: 1903.475000
   current CPU time: 1909.324000
   fraction done: 0.214406
   swap size: 58 MB
   working set size: 56 MB
3) -----------
   name: blc34_2bit_guppi_58406_27281_HIP20917_0104.20977.818.21.44.149.vlar_0
   WU name: blc34_2bit_guppi_58406_27281_HIP20917_0104.20977.818.21.44.149.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4643.434099
   CPU time at last checkpoint: 1902.425000
   current CPU time: 1904.189000
   fraction done: 0.204658
   swap size: 58 MB
   working set size: 56 MB
4) -----------
   name: blc34_2bit_guppi_58406_26949_HIP20491_0103.21476.0.21.44.21.vlar_1
   WU name: blc34_2bit_guppi_58406_26949_HIP20491_0103.21476.0.21.44.21.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4636.707813
   CPU time at last checkpoint: 1845.896000
   current CPU time: 1862.998000
   fraction done: 0.205810
   swap size: 54 MB
   working set size: 52 MB
5) -----------
   name: blc34_2bit_guppi_58406_28965_HIP20350_0109.21465.0.22.45.59.vlar_1
   WU name: blc34_2bit_guppi_58406_28965_HIP20350_0109.21465.0.22.45.59.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4665.477705
   CPU time at last checkpoint: 1781.304000
   current CPU time: 1805.969000
   fraction done: 0.200882
   swap size: 58 MB
   working set size: 56 MB
6) -----------
   name: blc34_2bit_guppi_58406_28625_HIP21029_0108.20913.818.21.44.210.vlar_1
   WU name: blc34_2bit_guppi_58406_28625_HIP21029_0108.20913.818.21.44.210.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4675.696451
   CPU time at last checkpoint: 1780.113000
   current CPU time: 1784.886000
   fraction done: 0.199132
   swap size: 58 MB
   working set size: 56 MB
7) -----------
   name: blc34_2bit_guppi_58406_28965_HIP20350_0109.21465.0.22.45.101.vlar_1
   WU name: blc34_2bit_guppi_58406_28965_HIP20350_0109.21465.0.22.45.101.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4705.804477
   CPU time at last checkpoint: 1711.990000
   current CPU time: 1768.862000
   fraction done: 0.193975
   swap size: 54 MB
   working set size: 52 MB
8) -----------
   name: blc34_2bit_guppi_58406_28625_HIP21029_0108.20913.818.21.44.254.vlar_1
   WU name: blc34_2bit_guppi_58406_28625_HIP21029_0108.20913.818.21.44.254.vlar
   project URL: http://setiathome.berkeley.edu/
   received: Tue Apr  2 13:43:18 2019
   report deadline: Sat May 25 18:42:59 2019
   ready to report: no
   state: downloaded
   scheduler state: scheduled
   active_task_state: EXECUTING
   app version num: 800
   resources: 1 CPU
   estimated CPU time remaining: 4928.870679
   CPU time at last checkpoint: 1422.330000
   current CPU time: 1466.627000
   fraction done: 0.155767
   swap size: 54 MB
   working set size: 52 MB

boinccmd – the management tool for the command line

Here are the options you can use in version 7.14.2:

[myuser@compute1 ~]# boinccmd --help

usage: boinccmd [--host hostname] [--passwd passwd] [--unix_domain] command

default hostname: localhost
default password: contents of gui_rpc_auth.cfg
Commands:
 --acct_mgr attach URL name passwd  attach to account manager
 --acct_mgr info                    show current account manager info
 --acct_mgr sync                    synchronize with acct mgr
 --acct_mgr detach                  detach from acct mgr
 --client_version                   show client version
 --create_account URL email passwd name
 --file_transfer URL filename op    file transfer operation
   op = retry | abort
 --get_app_config URL               show app config for given project
 --get_cc_status
 --get_daily_xfer_history           show network traffic history
 --get_disk_usage                   show disk usage
 --get_file_transfers               show file transfers
 --get_host_info
 --get_message_count                show largest message seqno
 --get_messages [ seqno ]           show messages > seqno
 --get_notices [ seqno ]            show notices > seqno
 --get_project_config URL
 --get_project_status               show status of all attached projects
 --get_proxy_settings
 --get_simple_gui_info              show status of projects and active tasks
 --get_state                        show entire state
 --get_tasks                        show tasks
 --get_old_tasks                    show reported tasks from last 1 hour
 --join_acct_mgr URL name passwd    same as --acct_mgr attach
 --lookup_account URL email passwd
 --network_available                retry deferred network communication
 --project URL op                   project operation
   op = reset | detach | update | suspend | resume | nomorework | allowmorework | detach_when_done | dont_detach_when_done
 --project_attach URL auth          attach to project
 --quit                             tell client to exit
 --quit_acct_mgr                    same as --acct_mgr detach
 --read_cc_config
 --read_global_prefs_override
 --run_benchmarks
 --set_gpu_mode mode duration       set GPU run mode for given duration
   mode = always | auto | never
 --set_host_info product_name
 --set_network_mode mode duration   set network mode for given duration
   mode = always | auto | never
 --set_proxy_settings
 --set_run_mode mode duration       set run mode for given duration
   mode = always | auto | never
 --task url task_name op            task operation
   op = suspend | resume | abort

More clients in EPEL repository

[myuser@compute1 ~]# yum search boinc
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.checkdomain.de
 * epel: mirror.wiuwiu.de
 * extras: mirror.checkdomain.de
 * updates: mirror.checkdomain.de
============================================================================ N/S matched: boinc ============================================================================
boinc-client.x86_64 : The BOINC client
boinc-client-devel.x86_64 : Development files for boinc-client
boinc-client-doc.noarch : Documentation files for boinc-client
boinc-client-static.x86_64 : Static libraries for boinc-client
boinc-manager.x86_64 : GUI to control and monitor boinc-client

  Name and summary matches only, use "search all" for everything.

Troubleshooting

If you get the follwoing error:

root@srv ~ # boinccmd --project_attach "http://boinc.bakerlab.org/rosetta/" "1111_111111111111111111111111111"
Operation failed: authentication error

You must change the directory to “/var/lib/boinc” and run the command again. You may chown the directory to the boinc user. A directory for the project is created under “projects/” and a configuration file is created with name “account_boinc.bakerlab.org_rosetta.xml”.

root@srv ~ # cd /var/lib/boinc
root@srv /var/lib/boinc # boinccmd --project_attach "https://boinc.bakerlab.org/rosetta/" "1111_111111111111111111111111111"
root@srv chown -R boinc:boinc /var/lib/boinc
root@srv /var/lib/boinc # ls -altr projects/boinc.bakerlab.org_rosetta/
total 8
drwxrwx--x 4 boinc boinc 4096 Apr  4 03:01 ..
drwxrwx--x 2 boinc boinc 4096 Apr  4 03:01 .
root@srv /var/lib/boinc # cat account_boinc.bakerlab.org_rosetta.xml
<account>
    <master_url>https://boinc.bakerlab.org/rosetta/</master_url>
    <authenticator>1111_111111111111111111111111111</authenticator>
    <project_name>Rosetta@home</project_name>
<project_preferences>

<resource_share>15</resource_share>
<project_specific>
<color_scheme>Tahiti Sunset</color_scheme>
</project_specific>
</project_preferences>
<gui_urls>

    <gui_url>
        <name>Message boards</name>
        <description>Correspond with other users on the Rosetta@home message boards</description>
        <url>http://boinc.bakerlab.org/rosetta/forum_index.php</url>
    </gui_url>
    <gui_url>
        <name>Your account</name>
        <description>View your account information</description>
        <url>http://boinc.bakerlab.org/rosetta/home.php</url>
    </gui_url>
    <gui_url>
        <name>Your tasks</name>
        <description>View the last week or so of computational work</description>
        <url>http://boinc.bakerlab.org/rosetta/results.php?userid=xxxx</url>
    </gui_url>
    
</gui_urls>
</account>

how to disable effectively the deleting (purging) files from nginx proxy_cache (nginx cache manager process)

In peaks deleting files could kill your server and easily the traffic could degraded multiple times than normal if the nginx cache manager start deleting files!

The server is perfectly normal but suddenly it just get loaded and all nginx processes are in D (“Disk sleep”) state.

What could it be? What is going on with your proxy server?

Probably the cache is full!

Unfortunately there is no way to check how much is filled the cache live – just an upgrade or restart of the nginx process will trigger nginx cache loader to check all the cache files and will write the cache size on exit in the error log – but be careful the cache loading is also IO intensive operation – stats all the cache files and they could be millions images).

If you are sure the cache manager is to blame for the IO of your server (probably using this method – Check whether nginx cache manager is deleting files at the moment), you can stop it almost immediately!

Just increase the nginx cache drastically – add zero to the maximum cache size

Of course, you should have enough free space till you resolve the problem – for example more servers or manual deletion on peak-off or tune your cache deletion or any other solution….
Search for something like

        proxy_cache_path /mnt/cache levels=1:2 keys_zone=CACHESTATICS:900m inactive=710h max_size=400g

And add zero to the max_size number like:

        proxy_cache_path /mnt/cache levels=1:2 keys_zone=CACHESTATICS:900m inactive=710h max_size=4000g

The max size will increase from 400G to 4000G (4T)!
This will effectively stop the files deleting and the nginx cache manager will have slept for long time before invoking again to delete files. This could be life saving operation for your server at peak!

Here is a real graph from one of our servers – the cache manager started deleting files from the cache and the traffic dropped 99%!!!

SCREENSHOT 1) The nginx cache manager just started to delete files from the cache and this operation just killed our server completely.

You can see almost zero bandwidth! The problem was resolved when we reloaded nginx with a bigger cache max_size value. The nginx manager immediately went to sleep and no IO for deleting files. The load of the server returned to normal!

main menu
nginx cache manager start deleting files

SCREENSHOT 2) Hard drives were saturated and the disk maxed the IO time to 10 ms.

Despite the bigger READ and WRITE IOPS there was 95-99% less traffic.

main menu
Disk IO Time when cache manager is working

Then you can tune the values for deleting files from the cache – Tune nginx proxy cache – control the cache manager how to delete cached files.

Access Violation error when compiling packages in Gentoo – symlink

Here is another example of an Access violation error when building packages in Gentoo. This time the build process could not make a symbolic link in “/usr/bin” and the build process of the package failed with:

 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied

A detail explanation is available in our first article on the subject here – Access Violation error, when compiling packages in Gentoo.
All packages are built in a sandbox and there is a sandbox configuration in

/etc/sandbox.d/00default

, which instruct the build process where could write. If you get such an error in 99.99% there is a bug in the package and if you do not want to wait for fixing it (report it!) you can manually edit the SANDBOX_WRITE variable and add the path, which causes the build failure. Build the package and remove the added path!!! Or you risk making your system less secure!

We have problem with building the package “net-libs/srt-1.3.1”

srv1 src # emerge -v net-libs/srt

These are the packages that would be merged, in order:

[ebuild  N     ] net-libs/srt-1.3.1::gentoo  USE="-doc -gnutls -libressl" ABI_X86="32 (64) (-x32)" 0 KiB

Total: 1 package (1 new), Size of downloads: 0 KiB

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) net-libs/srt-1.3.1::gentoo
 * srt-1.3.1.tar.gz BLAKE2B SHA512 size ;-) ...                                                                                                        [ ok ]
>>> Unpacking source...
.....
.....
>>> Install srt-1.3.1 into /var/tmp/portage/net-libs/srt-1.3.1/image/ category net-libs
 * abi_x86_32.x86: running multilib-minimal_abi_src_install
>>> Working in BUILD_DIR: "/var/tmp/portage/net-libs/srt-1.3.1/work/srt-1.3.1-abi_x86_32.x86"
make -j6 -l10 install 
[ 28%] Built target haicrypt_virtual
[ 40%] Built target srtsupport_virtual
[ 80%] Built target srt_virtual
[ 83%] Built target srt_static
[ 85%] Built target srt_shared
[ 90%] Built target srt-file-transmit
[ 95%] Built target srt-live-transmit
[100%] Built target srt-multiplex
Install the project...
-- Install configuration: "Gentoo"
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so.1.3.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.a
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/version.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/logging_api.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/platform_sys.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt4udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/pkgconfig/haisrt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/pkgconfig/srt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied
-- Created symlink: /usr/bin/stransmit -> srt-live-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-ffplay
 * abi_x86_64.amd64: running multilib-minimal_abi_src_install
>>> Working in BUILD_DIR: "/var/tmp/portage/net-libs/srt-1.3.1/work/srt-1.3.1-abi_x86_64.amd64"
make -j6 -l10 install 
[ 11%] Built target srtsupport_virtual
[ 52%] Built target srt_virtual
[ 80%] Built target haicrypt_virtual
[ 83%] Built target srt_static
[ 85%] Built target srt_shared
[ 90%] Built target srt-multiplex
[ 95%] Built target srt-file-transmit
[100%] Built target srt-live-transmit
Install the project...
-- Install configuration: "Gentoo"
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so.1.3.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.a
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/version.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/logging_api.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/platform_sys.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/udt.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt4udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/pkgconfig/haisrt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/pkgconfig/srt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied
-- Created symlink: /usr/bin/stransmit -> srt-live-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-ffplay
>>> Completed installing srt-1.3.1 into /var/tmp/portage/net-libs/srt-1.3.1/image/

 * Final size of build directory: 14632 KiB (14.2 MiB)
 * Final size of installed tree:   5324 KiB ( 5.1 MiB)

 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/log/sandbox/sandbox-25570.log"
 * 
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: symlink
S: deny
P: /usr/bin/stransmit
A: /usr/bin/stransmit
R: /usr/bin/stransmit
C: /usr/bin/cmake -E create_symlink srt-live-transmit /usr/bin/stransmit 

F: symlink
S: deny
P: /usr/bin/stransmit
A: /usr/bin/stransmit
R: /usr/bin/stransmit
C: /usr/bin/cmake -E create_symlink srt-live-transmit /usr/bin/stransmit 
 * --------------------------------------------------------------------------------

>>> Failed to emerge net-libs/srt-1.3.1, Log file:

>>>  '/var/tmp/portage/net-libs/srt-1.3.1/temp/build.log'

In the installation phase occurred the package failure leaving half installed package. So we edited the “/etc/sandbox.d/00default” and added “:/usr/bin” at the end of SANDBOX_WRITE:

SANDBOX_WRITE="/usr/tmp/conftest:/usr/lib/conftest:/usr/lib32/conftest:/usr/lib64/conftest:/usr/tmp/cf:/usr/lib/cf:/usr/lib32/cf:/usr/lib64/cf:/usr/bin"

Then rebuild the package with emerge and remove the added path “:/usr/bin”. This is a dirty workaround, but it will allow you to use srt (and as a dependency to another packages’ installation).

LSI MegaRAID 2108 freezes with abort command and all processes hang up in disk sleep

It happened to one of our old LSI MegaRAID 2108 controllers (AOC-USAS2LP-H8iR (smc2108) with 36 disk, 32x2T and 4x8T) to freeze and most of the processes hang up with Disk sleep. The server was up, the network was working, but no login could be successful. A hard reset was executed with the IPMI KVM. The server started up, the MegaRAID controller booted with a warning that it was shutdown unexpectedly so there could be possible loss of data and to accept it by pressing any key or “C” to boot in the WebBIOS of the controller.

To summarize it up: the LSI controller hangs up when is in the following modes:

  1. Background Initialization
  2. Check Consistency

Aborting and disabling the modes above let out controller to work till replacement. If you experience any kind of strange disk hangs or freezes you can try our solution here! Check below to see how to do it yourself.

Keep on reading!

How to compile xmr-stak (2.5) under CentOS 6 for CPU mining cryptocurrencies in November 2018

In continuing our crypto series with xmr-stak we update the old article How to compile xmr-stak (2.4.5) under CentOS 7 for CPU mining cryptocurrencies with the new version 2.5 and tested the build procedure with the old CentOS 6.

In crypto mining it is important to have latest version of the mining software, because crypto world is super dynamic with thousands crypto currency and tens of algos, which ever month might be changed and updated!

Thanks to xmr-stak we can have one application capable of mining many different cryptocurrencies based on different algorithms. XMR-STAK is GPU and CPU miner, here we present only the CPU ability. Here are the steps you should do to install (in fact install dependencies and compile the miner) and to begin CPU mining:

STEP 1) Update your system and install the following dependencies

Because xmr-stak need a newer version of GNU GCC than the CentOS ships with you must install GNU GCC 5.1 at least. Because the latest versions have better optimizations for the processors we use here GNU GCC 7 from Software Collection repository – more information for this repository here: How to install new gcc and development tools under CentOS 7

[root@srv ~]# yum update -y
[root@srv ~]# yum install -y centos-release-scl epel-release
[root@srv ~]# yum install -y cmake3 devtoolset-7-gcc* hwloc-devel libmicrohttpd-devel openssl openssl-devel make git screen

STEP 2) Compile the mining program xmr-stak

[root@srv ~]# scl enable devtoolset-7 bash
[root@srv ~]# git clone https://github.com/fireice-uk/xmr-stak.git
Initialized empty Git repository in /root/xmr-stak/.git/
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 6002 (delta 0), reused 1 (delta 0), pack-reused 6001
Receiving objects: 100% (6002/6002), 1.86 MiB | 1.02 MiB/s, done.
Resolving deltas: 100% (4283/4283), done.
[root@srv ~]# mkdir xmr-stak/build
[root@srv ~]# cd xmr-stak/build
[root@srv build]# export CFLAGS="-O2 -march=native -msse3 -fomit-frame-pointer -pipe"
[root@srv build]# export CHOST="x86_64-pc-linux-gnu"
[root@srv build]# export CXXFLAGS="${CFLAGS}"
[root@srv build]# export LDFLAGS="-Wl,-O1"
[root@srv build]# cmake3 .. -DCUDA_ENABLE=OFF -DOpenCL_ENABLE=OFF
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc
-- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found OpenSSL: /usr/lib64/libssl.so;/usr/lib64/libcrypto.so (found version "1.0.1e") 
-- The ASM compiler identification is GNU
-- Found assembler: /opt/rh/devtoolset-7/root/usr/bin/cc
-- Configuring done
-- Generating done
-- Build files have been written to: /root/xmr-stak/build
[root@srv build]# make -j 8
Scanning dependencies of target xmr-stak-asm
[  3%] Building ASM object CMakeFiles/xmr-stak-asm.dir/xmrstak/backend/cpu/crypto/asm/cryptonight_v8_main_loop.S.o
[  6%] Linking C static library bin/libxmr-stak-asm.a
[  6%] Built target xmr-stak-asm
Scanning dependencies of target xmr-stak-c
[ 10%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_blake256.c.o
[ 13%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_jh.c.o
[ 20%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_keccak.c.o
[ 20%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_groestl.c.o
[ 24%] Building C object CMakeFiles/xmr-stak-c.dir/xmrstak/backend/cpu/crypto/c_skein.c.o
[ 27%] Linking C static library bin/libxmr-stak-c.a
[ 27%] Built target xmr-stak-c
Scanning dependencies of target xmr-stak-backend
[ 34%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/jconf.cpp.o
[ 34%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/cpuType.cpp.o
[ 37%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/jconf.cpp.o
[ 44%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/version.cpp.o
[ 44%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/hwlocMemory.cpp.o
[ 55%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/minethd.cpp.o
[ 55%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/backendConnector.cpp.o
[ 55%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/globalStates.cpp.o
[ 58%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/backend/cpu/crypto/cryptonight_common.cpp.o
[ 62%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/http/httpd.cpp.o
[ 65%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/http/webdesign.cpp.o
[ 68%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/console.cpp.o
[ 72%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/executor.cpp.o
[ 75%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/telemetry.cpp.o
[ 79%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/uac.cpp.o
[ 82%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/misc/utility.cpp.o
[ 86%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/net/jpsock.cpp.o
[ 89%] Building CXX object CMakeFiles/xmr-stak-backend.dir/xmrstak/net/socket.cpp.o
[ 93%] Linking CXX static library bin/libxmr-stak-backend.a
[ 93%] Built target xmr-stak-backend
Scanning dependencies of target xmr-stak
[ 96%] Building CXX object CMakeFiles/xmr-stak.dir/xmrstak/cli/cli-miner.cpp.o
[100%] Linking CXX executable bin/xmr-stak
[100%] Built target xmr-stak

Execute the program to see if everything is OK. With the help output you can see how many algorithms are supported as of version 2.5 of xmr-stak:

[root@srv build]# cd bin
[root@srv bin]# ./xmr-stak --help
Usage: xmr-stak [OPTION]...
 
  -h, --help                 show this help
  -v, --version              show version number
  -V, --version-long         show long version number
  -c, --config FILE          common miner configuration file
  -C, --poolconf FILE        pool configuration file
  --benchmark BLOCKVERSION   ONLY do a benchmark and exit
  --benchwait WAIT_SEC             ... benchmark wait time
  --benchwork WORK_SEC             ... benchmark work time
  --noCPU                    disable the CPU miner backend
  --cpu FILE                 CPU backend miner config file
  -i --httpd HTTP_PORT       HTTP interface port
 
The following options can be used for automatic start without a guided config,
If config exists then this pool will be top priority.
  -o, --url URL              pool url and port, e.g. pool.usxmrpool.com:3333
  -O, --tls-url URL          TLS pool url and port, e.g. pool.usxmrpool.com:10443
  -u, --user USERNAME        pool user name or wallet address
  -r, --rigid RIGID          rig identifier for pool-side statistics (needs pool support)
  -p, --pass PASSWD          pool password, in the most cases x or empty ""
  --use-nicehash             the pool should run in nicehash mode
  --currency NAME            currency to mine

Supported coin options: 
        - aeon7
        - bbscoin
        - bittube
        - cryptonight
        - cryptonight_bittube2
        - cryptonight_masari
        - cryptonight_haven
        - cryptonight_heavy
        - cryptonight_lite
        - cryptonight_lite_v7
        - cryptonight_lite_v7_xor
        - cryptonight_v7
        - cryptonight_v8
        - cryptonight_v7_stellite
        - graft
        - haven
        - intense
        - masari
        - monero
        - qrl
        - ryo
        - stellite
        - turtlecoin

Version: xmr-stak 2.5.2 752fd1e
Brought to by fireice_uk and psychocrypt under GPLv3.