List all your files (and directories) with file size over FTP without ls -R (recursive)

A great piece of software is

lftp – sophisticated file transfer program

This little console tool could ease your life significantly with many enhancements to the simple FTP protocol. This tip is for those how what to list all their files in a directory or the entire FTP account, but do not have ls command with recursive abilities. So the only option is to manually go through all the directories to fetch the listing information of the directories, but this could be automatically done by

lftp using the custom command “find” and if you add “-l” argument the output is like “ls -al” – file or directory, file permissions, user and group, file size, date and file name are shown on single line for each file.

Just execute the command with proper credentials and the starting directory of your choice. The command output could even be piped to another command.
Keep on reading!

MariaDB/MySQL replication error – Error during XID COMMIT: failed to update GTID state in mysql.gtid_slave_pos

When in aggressive parallel mode MariaDB/MySQL replication could fail with:

Last_Errno: 1942
Last_Error: Error during XID COMMIT: failed to update GTID state in mysql.gtid_slave_pos: 1062: Duplicate entry '0-46158188501' for key 'PRIMARY'

This table is used for tracking the replication process and you might probably just do:

STOP/START SLAVE i.e. restart the replication and it would continue without errors.

MariaDB [(none)]> STOP SLAVE;
Query OK, 0 rows affected (0.08 sec)

MariaDB [(none)]> START SLAVE;
Query OK, 0 rows affected (0.00 sec)

Optimistic or aggressive mode runs conflicting transactions in parallel and it sometimes happens to roll back. In our case probably something happened and the rollback failed and STOP/START saved the replication.

* Additional thoughts

If you try STOP/START and you get the same error, probably it worth trying truncating the table “mysql.gtid_slave_pos” if you do not use GTID Replication feature (the “show slave status” says “Using_Gtid: No”). And even if you use “Using_Gtid: No” you could probably always stop the replication, “change master” to use the old style and start again? Probably switching off the aggressive mode might help, too!
Keep on reading!

Smart Array P440 – create RAID 1+0 (RAID 10) using Smart Storage Administrator

This article is to show how to create RAID 1+0 in Smart Array P440 hardware controller and what kind of migration is possible from RAID 1+0 on this controller.

Existing RAID 1+0 could be migrated to RAID 0 or RAID 5 or RAID 6 (so the RAID Level transformation is possible) with different Stip size (any of the supported in the controller) on-the-fly with no data loss!

No data loss is available to the tested server HP ProLiant DL160 GEN9, check the manual for your generation.
You may check our more detail article on how to start HPE Smart Storage Administrator on your server here: Review of Smart Array P440 on a server HP ProLiant DL160 Gen9 using iLO – create, modify, delete array and view controller settings

There are the steps to create RAID 1+0 using Smart Storage Administrator:

STEP 1) Click on “Create Array” to create a new array.

main menu
Logical Devices

Keep on reading!

Smart Array P440 – create RAID 6 using Smart Storage Administrator

This article is to show how to create RAID 6 (ADG) in Smart Array P440 hardware controller and what kind of migration is possible from RAID 6 on this controller.

Existing RAID 6 could be migrated to RAID 0 or RAID 5 or RAID 1+0 (so the RAID Level transformation is possible) with different Stip size (any of the supported in the controller) on-the-fly with no data loss!

No data loss is available to the tested server HP ProLiant DL160 GEN9, check the manual for your generation.
You may check our more detail article on how to start HPE Smart Storage Administrator on your server here: Review of Smart Array P440 on a server HP ProLiant DL160 Gen9 using iLO – create, modify, delete array and view controller settings

There are the steps to create RAID 6 (ADG) using Smart Storage Administrator:

STEP 1) Click on “Create Array” to create a new array.

main menu
Logical Devices

Keep on reading!

Smart Array P440 – create RAID 5 using Smart Storage Administrator

This article is to show how to create RAID 5 in Smart Array P440 hardware controller and what kind of migration is possible from RAID 5 on this controller.

Existing RAID 5 could be migrated to RAID 0 or RAID 5 (so the RAID Level transformation is possible) with different Stip size (any of the supported in the controller) on-the-fly with no data loss!

No data loss is available to the tested server HP ProLiant DL160 GEN9, check the manual for your generation.
You may check our more detail article on how to start HPE Smart Storage Administrator on your server here: Review of Smart Array P440 on a server HP ProLiant DL160 Gen9 using iLO – create, modify, delete array and view controller settings

There are the steps to create RAID 5 using Smart Storage Administrator:

STEP 1) Click on “Create Array” to create a new array.

main menu
Logical Devices

Keep on reading!

Smart Array P440 – create RAID 0 using Smart Storage Administrator

This article is to show how to create RAID 0 in Smart Array P440 hardware controller and what kind of migration is possible from RAID 0 on this controller.

Existing RAID 0 could be migrated only to the same RAID 0 (so no possible RAID Level transformation) with different Stip size (any of the supported in the controller) on-the-fly with no data loss!

You may check our more detail article on how to start HPE Smart Storage Administrator on your server here: Review of Smart Array P440 on a server HP ProLiant DL160 Gen9 using iLO – create, modify, delete array and view controller settings

There are the steps to create RAID 0 using Smart Storage Administrator:

STEP 1) Click on “Create Array” to create a new array.

main menu
Logical Devices

Keep on reading!

Review of Smart Array P440 on a server HP ProLiant DL160 Gen9 using iLO – create, modify, delete array and view controller settings

This article is to present how you can connect to the iLO remote keyboard, mouse and video (KVM) to create, migrate or view the arrays in hardware controller Smart Array P440. You are going to see multiple screenshots of the graphical user interface of Hewlett Packard Enterprise

Smart Storage Administrator (3.10.3.0)

This is not a command line manual of the command line tool to manage the Smart Array P440. This is the HP’s GUI utility, which you can enter by pressing F10 on server start. So to enter it you must restart your server and you must have physical access to the server or remote access over the Internet with your browser to use the HP’s KVM (keyboard, mouse, video) branded iLO – HP Integrated Lights-Out. With HP iLO you can manage your server as if you are physically in front of it. Check out more here https://en.wikipedia.org/wiki/HP_Integrated_Lights-Out

Here are the steps you can do to manage your Smart Array P440 on a modern HP server – HP ProLiant DL160 Gen9. Our server is powered Off state. This is summery what you can see in the following screenshots: first, we open the web interface of the HP iLO by typing the IP of the iLO. Then we click on “Java Web Start” on the Overview page (it is accessible in the Remote Console menu, too). We are going to create an array and then go through most of the controller and application settings. Read ahead to see the whole process…

SCREENSHOT 1) The login page of the HPE iLO ProLiant – the HP’s KVM of your server.

Write your login credentials and click on “Log In”.

main menu
HPE iLO ProLiant web login page

Keep on reading!

bind – dns server queries statistics with statistics-file

There is an option “statistics-file” in the BIND9 configuration for query statistics. It will give you statistics for

  • Incoming Requests – total number of queries
  • Incoming Queries – queries by record type
  • Outgoing Queries – queries by record type per view
  • Name Server Statistics – extended queries statistics by network connection type (UDP, TCP or IPv4 and IPv6 interface), by type of answer (authoritative, non authoritative and so on) and more
  • Zone Maintenance Statistics – transfer and system queries
  • Resolver Statistics – recursive queries
  • Cache DB RRsets – cached resources records sets
  • Socket I/O Statistics – statistics numbers for UDP, TCP (for both IPv4 and IPV6) sockets and connections opened, closed, failure
  • Per Zone Query Statistics – so you can see how many queries you have for a zone in a view (and the transfers if the server is a slave)

In named.conf:

options {
....
    statistics-file "/var/log/named.stats";
    zone-statistics yes ;
....

But if you check in /var/log this file might be missing even your BIND server has been running for months!

This is because the statistics and the file is generated on request and is a snapshot at the moment you do the request

To request from the BIND server to generate the file is pretty easy:

root@srv ~ # rndc stats
root@srv ~ #

No standard output and you should have the stats file generated:

root@srv ~ # ls -altr /var/log/named.stats 
-rw-r--r-- 1 named named 174997 Apr  7 01:43 /var/log/named.stats

The generated requests are appended in the file with a UNIX timestamp.

....
--- Statistics Dump --- (1550561292)
+++ Statistics Dump +++ (1551233218)
....

Keep on reading!

Gentoo kde-frameworks/kdewebkit failed compilation with Qt5WebKit could not be found because dependency is required

Updating the KDE Plasma Desktop in our Gentoo workstations this time failed with

CMake Error at /usr/share/cmake/Modules/CMakeFindDependencyMacro.cmake:48 (find_package):
  Found package configuration file:

    /usr/lib64/cmake/Qt5WebKit/Qt5WebKitConfig.cmake

  but it set Qt5WebKit_FOUND to FALSE so package "Qt5WebKit" is considered to
  be NOT FOUND.  Reason given by package:

  Qt5WebKit could not be found because dependency is required to have exact
  version 5.11.x.

It was strange because the previous emerge included the QT upgrade from old 5.11.2 to 5.12.1 and this dependency should have been resolved properly before:

emerge -vau $(qlist -IC|grep dev-qt|sort|uniq)

But apparently despite that the emerge built all QT libraries in dependency order the “dev-qt/qtwebkit” was built against the old QT libraries. And this is what is saying the above error!

The solution is really simple just rebuild the dev-qt/qtwebkit

root@srv ~ # emerge -va dev-qt/qtwebkit

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-qt/qtwebkit-5.212.0_pre20180120:5/5.212::gentoo  USE="X geolocation hyphen jit multimedia opengl printsupport qml -gles2 -gstreamer -nsplugin 
-orientation -webp" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] yes

Keep on reading!

nginx with php fpm (fastcgi) and the warning – an upstream response is buffered to a temporary file /var/cache/nginx/fastcgi_temp

As the web grows and the technology advances the page size of the web sites also grows or just some times you might want to output a big chunk of data from your application server – PHP-FPM (but it could be any of another ruby, python, C, Django and more), for example.
Here is a fast configuration tip (note this is not the proxy-related warning!):

The default nginx buffers per CGI connection are too small

Here is what to do in your nginx configuration file:
First, look for a line “include /etc/nginx/fastcgi_params;” or similar and add or edit if they exist after this line:

        fastcgi_buffer_size 16k;
        fastcgi_buffers 32 16k;

Check out more for the buffers here http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_buffers
The warning should stop if it does not stop you can try raising them. It could consume more memory but could lower the IO usage of your disks and improve the performance of your site or whatever backend works!

Here is the warning in our nginx error logs. We got this warning when using php-fpm and the php output size was 325965 bytes (~320K).

2019/04/04 09:56:05 [warn] 24451#24451: *44269838 an upstream response is buffered to a temporary file /var/cache/nginx/fastcgi_temp/0/12/0019966120 while reading upstream, client: 10.10.10.10, server: srv17.srv.en, request: "GET /api/20140102/product HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "srv17.srv.en"
2019/04/04 09:56:07 [warn] 24451#24451: *44269849 an upstream response is buffered to a temporary file /var/cache/nginx/fastcgi_temp/2/12/0019966122 while reading upstream, client: 10.10.10.11, server: srv17.srv.en, request: "GET /api/20140102/product HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "srv17.srv.en"
2019/04/04 09:56:09 [warn] 24450#24450: *44269856 an upstream response is buffered to a temporary file /var/cache/nginx/fastcgi_temp/7/12/0019966127 while reading upstream, client: 10.10.10.12, server: srv17.srv.en, request: "GET /api/20140102/product HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "srv17.srv.en"