sysbench – FATAL: Too large position discovered in request

Here is a quick fix, which could take you a long time solving it even with Internet, because not so much information on the topic!
Let’s you want to run sysbench to check the IO of an SSD or NVME or your old hard drive? You run something like:

sysbench --test=fileio --file-total-size=4G --file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 --max-time=300 --file-extra-flags=direct --file-test-mode=rndrd prepare
sysbench --test=fileio --file-total-size=4G --file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 --max-time=300 --file-extra-flags=direct --file-test-mode=rndrd run
sysbench --test=fileio --file-total-size=4G --file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 --max-time=300 --file-extra-flags=direct --file-test-mode=rndrd cleanup

Total of 4G files split in 128000 files and each file should be 32K seems OK at first, but you got:

FATAL: Too large position discovered in request

And the solution is simple enough:

file-total-size = file-num * file-block-size

And put the file-total-size in the units you used for file-block-size to be sure there are no rounding!
To put in simple words:

the size of file data should evenly divide by the number of files

So the above will transform to:

ssysbench --test=fileio --file-total-size=4096000K --file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 --max-time=300 --file-extra-flags=direct --file-test-mode=rndrd prepare
ssysbench --test=fileio --file-total-size=4096000K --file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 --max-time=300 --file-extra-flags=direct --file-test-mode=rndrd run
ssysbench --test=fileio --file-total-size=4096000K --file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 --max-time=300 --file-extra-flags=direct --file-test-mode=rndrd cleanup

Here are the computes:

–file-total-size=4096000K = –file-num=128000 * –file-block-size=32K

Simple equation could save you a lot of time!

* Good run – no errors!

srv@local:/disks/1/test# sysbench --test=fileio --file-total-size=4096000K \
--file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 \
--max-time=300 --file-extra-flags=direct --file-test-mode=rndrd prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

128000 files, 32Kb each, 4000Mb total
Creating files for the test...
srv@local:/disks/1/test# echo 3 > /proc/sys/vm/drop_caches
srv@local:/disks/1/test# sysbench --test=fileio --file-total-size=4096000K \
--file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 \
--max-time=300 --file-extra-flags=direct --file-test-mode=rndrd run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 100

Extra file open flags: 16384
128000 files, 32Kb each
3.9062Gb total file size
Block size 32Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 99 times)
Done.

Operations performed:  4605591 Read, 0 Write, 0 Other = 4605591 Total
Read 140.55Gb  Written 0b  Total transferred 140.55Gb  (479.74Mb/sec)
15351.63 Requests/sec executed

Test execution summary:
    total time:                          300.0067s
    total number of events:              4605591
    total time taken by event execution: 29893.5378
    per-request statistics:
         min:                                  0.07ms
         avg:                                  6.49ms
         max:                                 40.43ms
         approx.  95 percentile:               7.06ms

Threads fairness:
    events (avg/stddev):           46055.9100/51.84
    execution time (avg/stddev):   298.9354/0.00


sysbench 0.4.12:  multi-threaded system evaluation benchmark

srv@local:/disks/1/test# sysbench --test=fileio --file-total-size=4096000K \
--file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 \
--max-time=300 --file-extra-flags=direct --file-test-mode=rndrd run
Removing test files...

* Bad run – with the error: FATAL: Too large position discovered in request

srv@local:/disks/1/test# sysbench --test=fileio --file-total-size=4096M \
--file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 \
--max-time=300 --file-extra-flags=direct --file-test-mode=rndrd prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

128000 files, 32Kb each, 4096M total
Creating files for the test...
srv@local:/disks/1/test# echo 3 > /proc/sys/vm/drop_caches
srv@local:/disks/1/test# sysbench --test=fileio --file-total-size=4096M \
--file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 \
--max-time=300 --file-extra-flags=direct --file-test-mode=rndrd run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 100

Extra file open flags: 16384
128000 files, 32.768Kb each
3.9999Gb total file size
Block size 32Kb
Number of random requests for random IO: 200000
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
FATAL: Too large position discovered in request!
(last message repeated 30 times)
FATAL: Failed to read file! file: 80714 pos: 2 errno = 22 ()
FATAL: Too large position discovered in request!
(last message repeated 67 times)
Done.

Operations performed:  1 Read, 0 Write, 0 Other = 1 Total
Read 32Kb  Written 0b  Total transferred 32Kb  (114.71Kb/sec)
    3.58 Requests/sec executed

Test execution summary:
    total time:                          0.2790s
    total number of events:              1
    total time taken by event execution: 0.0038
    per-request statistics:
         min:                                  3.79ms
         avg:                                  3.79ms
         max:                                  3.79ms
         approx.  95 percentile:         10000000.00ms

Threads fairness:
    events (avg/stddev):           0.0100/0.10
    execution time (avg/stddev):   0.0000/0.00
srv@local:/disks/1/test# sysbench --test=fileio --file-total-size=4096M \
--file-block-size=32K --file-num=128000 --num-threads=100 --max-requests=0 \
--max-time=300 --file-extra-flags=direct --file-test-mode=rndrd cleanup
Removing test files...

Leave a Reply

Your email address will not be published.