Install Nginx virtual host traffic status module – traffic information in nginx and more per server block and upstreams

This article is going to show how to compile and install the Nginx module – ngx_http_vhost_traffic_status.

The module gathers traffic information per the server blocks and upstream servers and shows information for Nginx proxy cache like used space.

In addition, the module shows the type of the Response – 1xx, 2xx, 3xx, 4xx, 5xx and total. So when if problems occur in a server block or an upstream server
This module nginx-module-vts offers really extended status information for your Nginx.
Here is one the status page of our web servers with 18 virtual hosts:

The status page shows all virtual hosts in section “Server zones” and all upstream servers for the FastCGI PHP backend servers.

Traffic, requests, and status codes are available. All data is available in JSON, too.

main menu
Traffic information in Nginx and more per server block and upstreams

Server zones information

  • Requests – Total, Requests/s, Time
  • Responses – 1xx, 2xx, 3xx, 4xx, 5xx, Total
  • Traffic – Sent, Received, Sent/s, Received/s
  • Cache – Miss, Bypass, Expired, Stale, Updating, Revalidated, Hit, Scarce, Total

In addition to the information above there are State, Response Time, Weight, MaxFails and FileTimeout for all the upstream servers. And for the Nginx proxy cache there are Size, Capacity (live information!) and all information above per zone!
Keep on reading!

PHP 7.2 or PHP 7.3 with mcrypt – manual build

Newer PHP versions do not include PHP mcrypt library. The mcrypt module was part of PHP 5 till 7.1, in which it was deprecated and removed in 7.2. If you open the php.net documentation for mcrypt PHP functions you will see:

This function has been DEPRECATED as of PHP 7.1.0 and REMOVED as of PHP 7.2.0. Relying on this function is highly discouraged.

The mcrypt module is now in PHP PECL (repository for PHP Extensions) in https://pecl.php.net/package/mcrypt. As you can see in the description, this is a legacy module, which

Provides bindings for the unmaintained libmcrypt.

, so it is strongly recommended to replace it with OpenSSL (for example).
Still, if you need this legacy module – mcrypt and :

You may want to manually build the mcrypt module for your current installed PHP. Of course, the generic dependencies are:

  1. libmcrypt and its headers (if the Linux distribution) splits the binary and the headers
  2. GNU GCC
  3. PHP 7.2+
  4. download the latest mcrypt module source from https://pecl.php.net/package/mcrypt. For example, now it is https://pecl.php.net/get/mcrypt-1.0.2.tgz
mkdir /root/mcrypt-php-module-manual
cd /root/mcrypt-php-module-manual
wget https://pecl.php.net/get/mcrypt-1.0.2.tgz
tar xzf mcrypt-1.0.2.tgz
cd mcrypt-1.0.2
phpize
aclocal
libtoolize --force
autoheader
autoconf
./configure
make
make install

Do not use “make -j N” (“make -j 8”, for example), because it may fail to compile.
Keep on reading!

Access Violation error when compiling packets in Gentoo – symlink

Here is another example of Access violation error when building packages in Gentoo. This time the build process could not make a symbolic link in “/usr/bin” and the build process of the package failed with:

 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied

A detail explanation is available in our first article on the subject here – Access Violation error, when compiling packets in Gentoo.
All packages are built in a sandbox and there is a sandbox configuration in

/etc/sandbox.d/00default

, which instruct the build process where could write. If you get such error in 99.99% there is a bug in the package and if you do not want to wait for fixing it (report it!) you can manually edit the SANDBOX_WRITE variable and add the path, which causes the build failure. Build the package and remove the added path!!! Or you risk to make your system less secure!

We have problem with building the package “net-libs/srt-1.3.1”

srv1 src # emerge -v net-libs/srt

These are the packages that would be merged, in order:

[ebuild  N     ] net-libs/srt-1.3.1::gentoo  USE="-doc -gnutls -libressl" ABI_X86="32 (64) (-x32)" 0 KiB

Total: 1 package (1 new), Size of downloads: 0 KiB

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) net-libs/srt-1.3.1::gentoo
 * srt-1.3.1.tar.gz BLAKE2B SHA512 size 😉 ...                                                                                                        [ ok ]
>>> Unpacking source...
.....
.....
>>> Install srt-1.3.1 into /var/tmp/portage/net-libs/srt-1.3.1/image/ category net-libs
 * abi_x86_32.x86: running multilib-minimal_abi_src_install
>>> Working in BUILD_DIR: "/var/tmp/portage/net-libs/srt-1.3.1/work/srt-1.3.1-abi_x86_32.x86"
make -j6 -l10 install 
[ 28%] Built target haicrypt_virtual
[ 40%] Built target srtsupport_virtual
[ 80%] Built target srt_virtual
[ 83%] Built target srt_static
[ 85%] Built target srt_shared
[ 90%] Built target srt-file-transmit
[ 95%] Built target srt-live-transmit
[100%] Built target srt-multiplex
Install the project...
-- Install configuration: "Gentoo"
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so.1.3.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.so
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/libsrt.a
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/version.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/logging_api.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/platform_sys.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt4udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/pkgconfig/haisrt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib32/pkgconfig/srt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied
-- Created symlink: /usr/bin/stransmit -> srt-live-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-ffplay
 * abi_x86_64.amd64: running multilib-minimal_abi_src_install
>>> Working in BUILD_DIR: "/var/tmp/portage/net-libs/srt-1.3.1/work/srt-1.3.1-abi_x86_64.amd64"
make -j6 -l10 install 
[ 11%] Built target srtsupport_virtual
[ 52%] Built target srt_virtual
[ 80%] Built target haicrypt_virtual
[ 83%] Built target srt_static
[ 85%] Built target srt_shared
[ 90%] Built target srt-multiplex
[ 95%] Built target srt-file-transmit
[100%] Built target srt-live-transmit
Install the project...
-- Install configuration: "Gentoo"
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so.1.3.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so.1
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.so
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/libsrt.a
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/version.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/logging_api.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/platform_sys.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/udt.h
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/include/srt/srt4udt.h
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/pkgconfig/haisrt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/lib64/pkgconfig/srt.pc
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-live-transmit
 * ACCESS DENIED:  symlink:      /usr/bin/stransmit
CMake Error: failed to create symbolic link '/usr/bin/stransmit': permission denied
-- Created symlink: /usr/bin/stransmit -> srt-live-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-file-transmit
-- Installing: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-multiplex
-- Up-to-date: /var/tmp/portage/net-libs/srt-1.3.1/image/usr/bin/srt-ffplay
>>> Completed installing srt-1.3.1 into /var/tmp/portage/net-libs/srt-1.3.1/image/

 * Final size of build directory: 14632 KiB (14.2 MiB)
 * Final size of installed tree:   5324 KiB ( 5.1 MiB)

 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/log/sandbox/sandbox-25570.log"
 * 
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: symlink
S: deny
P: /usr/bin/stransmit
A: /usr/bin/stransmit
R: /usr/bin/stransmit
C: /usr/bin/cmake -E create_symlink srt-live-transmit /usr/bin/stransmit 

F: symlink
S: deny
P: /usr/bin/stransmit
A: /usr/bin/stransmit
R: /usr/bin/stransmit
C: /usr/bin/cmake -E create_symlink srt-live-transmit /usr/bin/stransmit 
 * --------------------------------------------------------------------------------

>>> Failed to emerge net-libs/srt-1.3.1, Log file:

>>>  '/var/tmp/portage/net-libs/srt-1.3.1/temp/build.log'

In the installation phase occurred the package failure leaving half installed package. So we edited the “/etc/sandbox.d/00default” and added “:/usr/bin” at the end of SANDBOX_WRITE:

SANDBOX_WRITE="/usr/tmp/conftest:/usr/lib/conftest:/usr/lib32/conftest:/usr/lib64/conftest:/usr/tmp/cf:/usr/lib/cf:/usr/lib32/cf:/usr/lib64/cf:/usr/bin"

Then rebuild the package with emerge and remove the added path “:/usr/bin”. This is a dirty workaround, but it will allow you to use srt (and as a dependency to another packages’ installation).

Missing pkg-config results in configure error: possibly undefined macro

This small article is for you who wonder why a configure script under Linux failed with an error like:

configure.ac:204: error: possibly undefined macro: AC_MSG_ERROR
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:216: error: possibly undefined macro: AC_CHECK_LIB
configure.ac:328: error: possibly undefined macro: AC_CHECK_HEADER
autoreconf: /usr/bin/autoconf failed with exit status: 1
 FAILED 

So autoreconf failed with missing macros, but you just saw you had the latest version and no clue what is wrong with the code you tried to compile!
In most cases the problem is trivial – you are missing the

pkg-config

It is a helper tool used when compiling. It helps to have the correct compiler options and not to hard-coded them in your files. So search for this tool in your Linux distro and install it.

Under CentOS 7

yum install pkgconfig

Under Ubuntu/Debian

apt-get install pkg-config

Under Gentoo

emerge -v dev-util/pkgconfig

Build your own kernel under Ubuntu using mainline (latest) kernel

Here we will show how to build your own kernel under Ubuntu 16/17/18. We are going to use

mainline kernel

because most of the cases when we need to build our own is when we need some new feature presented in the new kernels. So Ubuntu mainline kernels are selected new kernels, which are packed in deb packets for simplicity and can be used to test new features, but they are not supported by Ubuntu teams and you should test them really carefully if you want to use them in production!
The mainline kernels are here: http://kernel.ubuntu.com/~kernel-ppa/mainline/. This is the main address, where you could download the latest kernels and some other legacy one if you have old system.
This howto is based on https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel and https://wiki.ubuntu.com/Kernel/MainlineBuilds – we summarize the commands needed to build your new and latest kernel and show you some typical mistakes and problems during the process.
To show you more information our purpose will be to build the one of the latest kernels with low latency features enabled (Ubuntu has such type of kernel configuration)! And here is one of the most important dependency, which is not explicitly included or even mentioned in most of the tutorials for building your own kernels –

you MUST include new or even latest Linux firmware package

All Linux distros have a package or packages containing files with different firmware for multiple devices supported by the kernel modules. Basically firmware is used by a kernel module and it can be a micro code (program), which is uploaded in the device or just complete the kernel module to function properly with a device. Recent days firmware is like a black box the manufacturer of the devices pack a proprietary code in a firmware blob and the source code is unknown to the community.

The first thing you should do is to choose the kernel version you want to build. We want version 4.17.x as more stable of the latest 4.18, which was released a few days weeks ago (let’s wait for a month after the initial release before using it!).
So the main address is here http://kernel.ubuntu.com/~kernel-ppa/mainline/ and the kernel files, which we are going to use to rebuild are here http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.17.19/
Keep on reading!

Access Violation error, when compiling packets in Gentoo

Sometimes if you try to emerge a package in Gentoo you could receive error in the configure phase of the compilation process. The example below is with the emerging the PHP – dev-lang/php-5.6.33:5.6::gentoo, but could happen with many other packages, which are rather old and probably not maintained or the sandbox or even the portage packages are old.
So here is the error and the compilation stops:

srv ~ # emerge -av --nodeps "<php-7"
...
checking for mmap() using MAP_ANON shared memory support... yes
checking for mmap() using /dev/zero shared memory support... yes
checking for mmap() using shm_open() shared memory support...  * ACCESS DENIED:  open_wr:      /run/test.shm.8811LBKone
no
checking for mmap() using regular file shared memory support... yes
...
checking for mmap() using MAP_ANON shared memory support... yes
checking for mmap() using /dev/zero shared memory support... yes
checking for mmap() using shm_open() shared memory support...  * ACCESS DENIED:  open_wr:      /run/test.shm.180309hAMbj
no
checking for mmap() using regular file shared memory support... yes
....
Thank you for using PHP.
config.status: creating php5.spec
config.status: creating main/build-defs.h
config.status: creating scripts/phpize
config.status: creating scripts/man1/phpize.1
config.status: creating scripts/php-config
config.status: creating scripts/man1/php-config.1
config.status: creating ext/phar/phar.1
config.status: creating ext/phar/phar.phar.1
config.status: creating main/php_config.h
config.status: executing libtool commands
config.status: executing default commands
>>> Source configured.
 * --------------------------- ACCESS VIOLATION SUMMARY ---------------------------
 * LOG FILE: "/var/log/sandbox/sandbox-13466.log"
 * 
VERSION 1.0
FORMAT: F - Function called
FORMAT: S - Access Status
FORMAT: P - Path as passed to function
FORMAT: A - Absolute Path (not canonical)
FORMAT: R - Canonical Path
FORMAT: C - Command Line

F: open_wr
S: deny
P: /run/test.shm.21532Xx6ViE
A: /run/test.shm.21532Xx6ViE
R: /run/test.shm.21532Xx6ViE
C: ./conftest 

F: open_wr
S: deny
P: /run/test.shm.31817hurGxH
A: /run/test.shm.31817hurGxH
R: /run/test.shm.31817hurGxH
C: ./conftest 

F: open_wr
S: deny
P: /run/test.shm.8811LBKone
A: /run/test.shm.8811LBKone
R: /run/test.shm.8811LBKone
C: ./conftest 

F: open_wr
S: deny
P: /run/test.shm.180309hAMbj
A: /run/test.shm.180309hAMbj
R: /run/test.shm.180309hAMbj
C: ./conftest 
 * --------------------------------------------------------------------------------

>>> Failed to emerge dev-lang/php-5.6.33, Log file:

>>>  '/var/tmp/portage/dev-lang/php-5.6.33/temp/build.log'

You could try adding “-sandbox” to feature in “/etc/portage/make.conf”

FEATURES="-sandbox"

But

the sandbox feature is very important and should not be disabled by default.

And that’s why sometime when you disable it with “-sandbox” you still get access violation and you still cannot install/compile the package!
The thing is you see the error and you can fix it easily. The important part is the directory, which causes the error, in the above example with “dev-lang/php”, but could be any other Gentoo package, the problem is the writing permission for files in “/run” directory. So open the configuration file

/etc/sandbox.d/00default

and you’ll see the there is a variable called SANDBOX_WRITE, which accept paths. If you add to this variable at the end the directory “/run” or your access violated directory you’ll be able to install/compile your package with no problems, for the above problem the solution was:

SANDBOX_WRITE="/usr/tmp/conftest:/usr/lib/conftest:/usr/lib32/conftest:/usr/lib64/conftest:/usr/tmp/cf:/usr/lib/cf:/usr/lib32/cf:/usr/lib64/cf:/run"