bacula – Device not in SD Device resources or no matching Media Type

If you happen to have such error in your bacula backup server:

20-Jun 00:19 bkp-srv-dir JobId 23741: No prior Full backup Job record found.
20-Jun 00:19 bkp-srv-dir JobId 23741: No prior or suitable Full backup found in catalog. Doing FULL backup.
20-Jun 00:19 bkp-srv-dir JobId 23741: Start Backup JobId 23741, Job=srv1-img.2018-06-20_00.19.09_56
20-Jun 00:19 bkp-srv-sd JobId 23741: Fatal error: Device reservation failed for JobId=23741: 
20-Jun 00:19 bkp-srv-dir JobId 23741: Fatal error: 
     Storage daemon didn't accept Device "srv1-img" because:
     3924 Device "srv1-img" not in SD Device resources or no matching Media Type.
20-Jun 00:19 bkp-srv-dir JobId 23741: Error: Bacula bkp-srv-dir 7.0.5 (28Jul14):
  Build OS:               x86_64-pc-linux-gnu ubuntu 16.04
  JobId:                  23741
  Job:                    srv1-img.2018-06-20_00.19.09_56
  Backup Level:           Full (upgraded from Incremental)
  Client:                 "srv1-img" 7.4.0 (16Jan16) x86_64-pc-linux-gnu,ubuntu,12.04
  FileSet:                "srv1-img" 2018-04-13 23:05:00
  Pool:                   "srv1-img-full" (From Job FullPool override)
  Catalog:                "DefaultCatalog" (From Client resource)
  Storage:                "srv1-img" (From Job resource)
  Scheduled time:         20-Jun-2018 00:19:06
  Start time:             20-Jun-2018 00:19:11
  End time:               20-Jun-2018 00:19:11
  Elapsed time:           0 secs
  Priority:               10
  FD Files Written:       0
  SD Files Written:       0
  FD Bytes Written:       0 (0 B)
  SD Bytes Written:       0 (0 B)
  Rate:                   0.0 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               no
  Volume name(s):         
  Volume Session Id:      3105
  Volume Session Time:    1521814321
  Last Volume Bytes:      0 (0 B)
  Non-fatal FD errors:    1
  SD Errors:              0
  FD termination status:  
  SD termination status:  
  Termination:            *** Backup Error ***

First thing to check is the status of Storage Devices you have with bconsole:

bkp-srv bacula # bconsole 
Connecting to Director localhost:9101
1000 OK: 1 bkp-srv-dir Version: 7.0.5 (28 July 2014)
Enter a period to cancel a command.
*status storage
Automatically selected Storage: bkp-srv
Connecting to Storage daemon bkp-srv at 78.140.166.26:9103

bkp-srv-sd Version: 7.0.5 (28 July 2014) x86_64-pc-linux-gnu ubuntu 16.04
Daemon started 20-Jun-18 00:43. Jobs: run=1, running=0.
 Heap: heap=135,168 smbytes=302,033 max_bytes=478,209 bufs=243 max_bufs=262
 Sizes: boffset_t=8 size_t=8 int32_t=4 int64_t=8 mode=0,0

Running Jobs:
No Jobs running.
====

Jobs waiting to reserve a drive:
====

Terminated Jobs:
 JobId  Level    Files      Bytes   Status   Finished        Name 
===================================================================
 23736  Incr          3    383.9 K  OK       19-Jun-18 14:05 srv1-dns
 23737  Incr          3    859.9 K  OK       19-Jun-18 14:05 srv2-dns
 23738  Incr        485    301.8 M  OK       19-Jun-18 14:05 srv3-dns
 23739  Incr         12    91.44 M  OK       19-Jun-18 14:05 srv4-dns
 23740  Full          1    1.358 G  OK       19-Jun-18 14:14 BackupCatalog
 23741  Full          0         0   Error    20-Jun-18 14:19 srv1-img
====

Device status:

Device "localstorage" (/mnt/storage/backups/localstorage) is not open.
==

Device "srv1-dns" (/mnt/storage/backups/srv1-dns) is not open.
==

Device "srv2-dns" (/mnt/storage/backups/srv2-dns) is not open.
==

Device "srv3-dns" (/mnt/storage/backups/srv3-dns) is not open.
==

Device "srv4-dns" (/mnt/storage/backups/srv4-dns) is not open.
==

Device "bkp-srv" (/mnt/storage/backups/bkp-srv) is not open.
==
====

Used Volume status:
====

====

As you can see there is no storage Device with name “srv1-img” as it expected from the error above.
You should check two files:

bacula-dir.conf

and

bacula-sd.conf

In file “/etc/bacula/bacula-dir.conf” you should have section like this:

Storage {
  Name = srv1-img
  Address = 10.10.10.10
  SDPort = 9103
  Password = mysecretpass
  Device = srv1-img
  Media Type = File
  Maximum Concurrent Jobs = 10
}

And in file “/etc/bacula/bacula-sd.conf”:

Device {
   Name = srv1-img
   Media Type = File
   Archive Device = /mnt/storage/backups/srv1-img
   LabelMedia = yes;
   Random Access = Yes;
   AutomaticMount = yes;
   RemovableMedia = no;
   AlwaysOpen = no;
 }

The important parts for this particular error (“3924 Device “srv1-img” not in SD Device resources or no matching Media Type.”) are highlighted:

  1. “Name” – the name of the storage
  2. “Media Type” – the type of the storage

You may also check the path of the storage – /mnt/storage/backups/srv1-img.

The two parameters must be the same! And if you verify these two configurations exist in the files and the above two parameters are the same and still get the error you MUST restart your SD (storage device) daemon, a “reload” command from the bconsole won’t work!

bkp-srv bacula # systemctl restart bacula-sd.service
bkp-srv bacula # systemctl status bacula-sd.service
* bacula-sd.service - LSB: Start Bacula Storage Daemon at boot time
   Loaded: loaded (/etc/init.d/bacula-sd; bad; vendor preset: enabled)
   Active: active (running) since Wed 2018-06-20 00:43:22 UTC; 4s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 25249 ExecStop=/etc/init.d/bacula-sd stop (code=exited, status=0/SUCCESS)
  Process: 25259 ExecStart=/etc/init.d/bacula-sd start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/bacula-sd.service
           `-25269 /usr/sbin/bacula-sd -c /etc/bacula/bacula-sd.conf -u bacula -g tape

Jun 20 00:43:22 bkp-srv systemd[1]: Starting LSB: Start Bacula Storage Daemon at boot time...
Jun 20 00:43:22 bkp-srv bacula-sd[25259]:  * Starting Bacula Storage daemon... bacula-sd
Jun 20 00:43:22 bkp-srv bacula-sd[25259]:    ...done.
Jun 20 00:43:22 bkp-srv systemd[1]: Started LSB: Start Bacula Storage Daemon at boot time.

And verify it restarted successfully.

Then you the storage device named “srv1-img” should appear in bconsole status storage command:
First thing to check is the status of Storage Devices you have with bconsole:

bkp-srv bacula # bconsole 
Connecting to Director localhost:9101
1000 OK: 1 bkp-srv-dir Version: 7.0.5 (28 July 2014)
Enter a period to cancel a command.
*status storage
Automatically selected Storage: bkp-srv
Connecting to Storage daemon bkp-srv at 78.140.166.26:9103

bkp-srv-sd Version: 7.0.5 (28 July 2014) x86_64-pc-linux-gnu ubuntu 16.04
Daemon started 20-Jun-18 00:43. Jobs: run=1, running=0.
 Heap: heap=135,168 smbytes=302,033 max_bytes=478,209 bufs=243 max_bufs=262
 Sizes: boffset_t=8 size_t=8 int32_t=4 int64_t=8 mode=0,0

Running Jobs:
No Jobs running.
====

Jobs waiting to reserve a drive:
====

Terminated Jobs:
 JobId  Level    Files      Bytes   Status   Finished        Name 
===================================================================
 23736  Incr          3    383.9 K  OK       19-Jun-18 14:05 srv1-dns
 23737  Incr          3    859.9 K  OK       19-Jun-18 14:05 srv2-dns
 23738  Incr        485    301.8 M  OK       19-Jun-18 14:05 srv3-dns
 23739  Incr         12    91.44 M  OK       19-Jun-18 14:05 srv4-dns
 23740  Full          1    1.358 G  OK       19-Jun-18 14:14 BackupCatalog
 23741  Full          0         0   Error    20-Jun-18 14:19 srv1-img
====

Device status:

Device "localstorage" (/mnt/storage/backups/localstorage) is not open.
==

Device "srv1-dns" (/mnt/storage/backups/srv1-dns) is not open.
==

Device "srv2-dns" (/mnt/storage/backups/srv2-dns) is not open.
==

Device "srv3-dns" (/mnt/storage/backups/srv3-dns) is not open.
==

Device "srv4-dns" (/mnt/storage/backups/srv4-dns) is not open.
==

Device "bkp-srv" (/mnt/storage/backups/bkp-srv) is not open.
==

Device "srv1-img" (/mnt/storage/backups/srv1-img) is not open.
==
====

Used Volume status:
====

====

Extras – rerun the job and verify the backup was executed.

bkp-srv bacula # bconsole 
Connecting to Director localhost:9101
1000 OK: 1 bkp-srv-dir Version: 7.0.5 (28 July 2014)
Enter a period to cancel a command.
*run
Automatically selected Catalog: DefaultCatalog
Using Catalog "DefaultCatalog"
A job name must be specified.
The defined Job resources are:
     1: BackupCatalog
     2: srv1-dns
     3: srv2-dns
     4: srv3-dns
     5: srv4-dns
     6: srv1-img
Select Job resource (1-33): 6
Run Backup job
JobName:  srv1-img
Level:    Incremental
Client:   srv1-img
FileSet:  srv1-img
Pool:     Default (From Job resource)
Storage:  srv1-img (From Job resource)
When:     2018-06-20 00:43:59
Priority: 10
OK to run? (yes/mod/no): yes
Job queued. JobId=23742
You have messages.
*messages 
20-Jun 00:44 bkp-srv-dir JobId 23742: No prior Full backup Job record found.
20-Jun 00:44 bkp-srv-dir JobId 23742: No prior or suitable Full backup found in catalog. Doing FULL backup.
20-Jun 00:44 bkp-srv-dir JobId 23742: Start Backup JobId 23742, Job=srv1-img.2018-06-20_00.44.01_00
20-Jun 00:44 bkp-srv-dir JobId 23742: Created new Volume="srv1-img-full0394", Pool="srv1-img-full", MediaType="File" in catalog.
20-Jun 00:44 bkp-srv-dir JobId 23742: Using Device "srv1-img" to write.
20-Jun 00:44 bkp-srv-sd JobId 23742: Labeled new Volume "srv1-img-full0394" on file device "srv1-img" (/mnt/storage/backups/srv1-img).
20-Jun 00:44 bkp-srv-sd JobId 23742: Wrote label to prelabeled Volume "srv1-img-full0394" on file device "srv1-img" (/mnt/storage/backups/srv1-img)
*messages
20-Jun 00:44 bkp-srv-dir JobId 23742: Volume used once. Marking Volume "srv1-img-full0394" as Used.
*messages
You have no messages.

Bacula – show configuration, status and information with bconsole tool

The following list of commands could be used to get a brief or detailed view of a Bacula backup server from the management utility

bconsole

. These commands are extremely useful for getting information on the backup process and policy and Bacula troubleshooting – could be used for fast debugging of an error, problems or misconfiguration.

the following commands give information for

  • Jobs

    list jobtotals Lists stats for all jobs, it also shows all jobs’ names
    show jobs Lists all jobs with their full configurations – show all jobs and for each job show detail explanation of what represent. The detail output includes full configuration of a job including client, catalog, fileset,schedule,pool,message. This will show all relationships between the different components of bacula system, how and which clients,storages,pools,schedules,filesets relate to. You’ll a thoroughly view of how let’s say a server is made the backup.
    show job=[job_name] shows full configuration for a job, the name could be taken by the two above commands
    list jobs Lists all jobs’ status – ID, StartTime, Type (backup?), Level (Full, increment, differential?), Files and bytes processed and the status of the job (Terminated normally, Running,Fatal error and so on)
    list files jobid=[ID] which files were included in the backup? Lists all paths and files included in the backup, not a configuration set but real physical path and filenames.
    status dir if you want to find all scheduled jobs for the next day (or more if you add a parameter). This will show the status of the director process.
  • Storages

    status storage list storage devices and their status – you can see the physical path on the filesystem where the Devices will put backup files
  • Clients

    show client show all clients’ names and backup policy
    status client=[client_name] show client status and what is doing, check the network connection between the director and the client, last terminated jobs and their status.
  • Filesets

    show fileset show all filesets, a fileset is a set with files and directories to include or exclude from a backup.
  • Schedule

    show schedule shows all registered schedulers and details for each one (Run Level=Full,Differential,Incremental), months, days, minutes.
    show schedule=[scheduler_name] shows details for a schedule with name scheduler_name (Run Level=Full,Differential,Incremental), months, days, minutes. It’s like the schedule backup plan of a server
  • Director

    message shows the last message of the backup process. If empty all logs of the backup process could be found in “/var/log/bacula/bacula.log”
    reload the director will re-read its all configuration files. Should be used when adding configuration files.

And here is the example output of the above commands with a little bit of explanation:

  • Jobs

    1) Get the stats and the jobs’ names, the names could be used in many other commands!

    srvbkp@local # bconsole 
    Connecting to Director localhost:9101
    1000 OK: 1 srvbkp-dir Version: 7.0.5 (28 July 2014)
    Enter a period to cancel a command.
    *list jobtotals
    Automatically selected Catalog: allbackup
    Using Catalog "allbackup"
    +------+-----------+-------------------+--------------------------------+
    | Jobs | Files     | Bytes             | Job                            |
    +------+-----------+-------------------+--------------------------------+
    |   90 |        90 |   123,665,584,337 | BackupCatalog                  |
    |    5 |         5 |   281,593,737,603 | RestoreFiles                   |
    |   13 | 1,232,316 |   118,480,634,434 | srv1-media                     |
    |   32 |        12 |             3,674 | srv1-dns                       |
    |   32 |        10 |             3,064 | srv2-dns                       |
    |   32 |        10 |             3,064 | srv3-dns                       |
    |   32 |        10 |             3,086 | srv4-dns                       |
    |   32 |        10 |             3,084 | srv5-dns                       |
    |   26 | 3,837,536 |   587,812,183,466 | srv1-images                    |
    +------+-----------+-------------------+--------------------------------+
    +-------+------------+-------------------+
    | Jobs  | Files      | Bytes             |
    +-------+------------+-------------------+
    | 1,474 | 14,925,321 | 5,475,024,028,957 |
    +-------+------------+-------------------+
    *
    

    2) Get all configurations for all jobs, here are included only two for clarity. All needed information for taking a backup of a server. You can see the files, which will be included (or excluded), where the backup will be stored when will happen in time, and how many different types of backup will be done – full, incremental, and differential. And this whole information is for all clients (servers).

    *show jobs
    Job: name=srv1-dns JobType=66 level= Priority=10 Enabled=1
         MaxJobs=1 Resched=0 Times=0 Interval=1,800 Spool=0 WritePartAfterJob=1
         Accurate=0
      --> Client: name=srv1-dns address=192.168.0.100 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
      --> FileSet: name=bind
          O MZ6
          N
          I /var/lib/named
          N
      --> Schedule: name=bind
      --> Run Level=Full
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=0 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Differential
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=1 2 3 4 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Incremental
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=1 2 3 4 5 6 
          wom=0 1 2 3 4 5 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Storage: name=bind address=192.168.0.10 SDport=9103 MaxJobs=10
          DeviceName=bind MediaType=File StorageId=17
      --> Pool: name=Default PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=100 auto_prune=1 VolRetention=1 year 
          VolUse=0 secs recycle=1 LabelFormat=*None*
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=53687091200
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-full PoolType=Backup
          use_cat=1 use_once=1 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=2 months 
          VolUse=0 secs recycle=1 LabelFormat=bind-full
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-incr PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=7 days 
          VolUse=23 hours  recycle=1 LabelFormat=bind-incr
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-diff PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=1 month 1 day 
          VolUse=0 secs recycle=1 LabelFormat=bind-diff
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Messages: name=Standard
          mailcmd=/usr/sbin/bsmtp -h localhost -f "(Bacula) <%r>" -s "Bacula: %t %e of %c %l" %r
          opcmd=/usr/sbin/bsmtp -h localhost -f "(Bacula) <%r>" -s "Bacula: Intervention needed for %j" %r
    Job: name=srv2-dns JobType=66 level= Priority=10 Enabled=1
         MaxJobs=1 Resched=0 Times=0 Interval=1,800 Spool=0 WritePartAfterJob=1
         Accurate=0
      --> Client: name=srv2-dns address=192.168.0.101 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
      --> FileSet: name=bind
          O MZ6
          N
          I /var/lib/named
          N
      --> Schedule: name=bind
      --> Run Level=Full
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=0 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Differential
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=1 2 3 4 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Incremental
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=1 2 3 4 5 6 
          wom=0 1 2 3 4 5 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Storage: name=bind address=192.168.0.10 SDport=9103 MaxJobs=10
          DeviceName=bind MediaType=File StorageId=17
      --> Pool: name=Default PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=100 auto_prune=1 VolRetention=1 year 
          VolUse=0 secs recycle=1 LabelFormat=*None*
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=53687091200
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-full PoolType=Backup
          use_cat=1 use_once=1 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=2 months 
          VolUse=0 secs recycle=1 LabelFormat=bind-full
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-incr PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=7 days 
          VolUse=23 hours  recycle=1 LabelFormat=bind-incr
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-diff PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=1 month 1 day 
          VolUse=0 secs recycle=1 LabelFormat=bind-diff
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Messages: name=Standard
          mailcmd=/usr/sbin/bsmtp -h localhost -f "(Bacula) <%r>" -s "Bacula: %t %e of %c %l" %r
          opcmd=/usr/sbin/bsmtp -h localhost -f "(Bacula) <%r>" -s "Bacula: Intervention needed for %j" %r
    

    3) You can get the full configuration information of a job (the information is the same as above, but for a given job name, which could be taken from the first command above, it is not necessary to output all the configurations every time):

    srvbkp@local # bconsole 
    Connecting to Director localhost:9101
    1000 OK: 1 srvbkp-dir Version: 7.0.5 (28 July 2014)
    Enter a period to cancel a command.
    *show jobs=srv2-dns
    Job: name=srv2-dns JobType=66 level= Priority=10 Enabled=1
         MaxJobs=1 Resched=0 Times=0 Interval=1,800 Spool=0 WritePartAfterJob=1
         Accurate=0
      --> Client: name=srv2-dns address=192.168.0.101 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
      --> FileSet: name=bind
          O MZ6
          N
          I /var/lib/named
          N
      --> Schedule: name=bind
      --> Run Level=Full
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=0 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Differential
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=1 2 3 4 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Incremental
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=1 2 3 4 5 6 
          wom=0 1 2 3 4 5 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Storage: name=bind address=192.168.0.10 SDport=9103 MaxJobs=10
          DeviceName=bind MediaType=File StorageId=17
      --> Pool: name=Default PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=100 auto_prune=1 VolRetention=1 year 
          VolUse=0 secs recycle=1 LabelFormat=*None*
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=53687091200
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-full PoolType=Backup
          use_cat=1 use_once=1 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=2 months 
          VolUse=0 secs recycle=1 LabelFormat=bind-full
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-incr PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=7 days 
          VolUse=23 hours  recycle=1 LabelFormat=bind-incr
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Pool: name=bind-diff PoolType=Backup
          use_cat=1 use_once=0 cat_files=1
          max_vols=0 auto_prune=1 VolRetention=1 month 1 day 
          VolUse=0 secs recycle=1 LabelFormat=bind-diff
          CleaningPrefix=*None* LabelType=0
          RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
          MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=0
          MigTime=0 secs MigHiBytes=0 MigLoBytes=0
          JobRetention=0 secs FileRetention=0 secs
      --> Messages: name=Standard
          mailcmd=/usr/sbin/bsmtp -h localhost -f "(Bacula) <%r>" -s "Bacula: %t %e of %c %l" %r
          opcmd=/usr/sbin/bsmtp -h localhost -f "(Bacula) <%r>" -s "Bacula: Intervention needed for %j" %r
    

    4) Lists all jobs’ status – ID, StartTime, Type (backup?), Level (Full, increment, differential?), Files and bytes processed and the status of the job (Terminated normally, Running,Fatal error and so on). Found out if you have backups of a server or the backup process failed!

    srvbkp@local # bconsole 
    Connecting to Director localhost:9101
    1000 OK: 1 srvbkp-dir Version: 7.0.5 (28 July 2014)
    Enter a period to cancel a command.
    *list jobs
    +-------+--------------------------------+---------------------+------+-------+-----------+-----------------+-----------+
    | JobId | Name                           | StartTime           | Type | Level | JobFiles  | JobBytes        | JobStatus |
    +-------+--------------------------------+---------------------+------+-------+-----------+-----------------+-----------+
    |   128 | srv1-test                      | 2016-12-04 23:05:00 | B    | F     |    17,506 |      52,116,400 | T         |
    |   178 | srv1-test                      | 2016-12-09 23:05:01 | B    | I     |        13 |           1,509 | T         |
    |   188 | srv1-test                      | 2016-12-10 23:05:01 | B    | I     |        13 |           1,509 | T         |
    .........................................................................................................................
    | 8,927 | srv2-images                    | 2018-03-04 20:00:00 | B    | F     |         0 |               0 | f         |
    | 8,928 | srv1-media                     | 2018-03-04 20:00:00 | B    | F     |         3 |             978 | T         |
    | 8,930 | srv1-dns                       | 2018-03-04 20:00:01 | B    | F     |         6 |           1,843 | T         |
    | 8,932 | srv2-dns                       | 2018-03-04 20:00:01 | B    | F     |         6 |           1,837 | T         |
    | 8,931 | srv3-dns                       | 2018-03-04 20:00:03 | B    | F     |         5 |           1,542 | T         |
    | 8,933 | srv4-dns                       | 2018-03-04 20:00:04 | B    | F     |         4 |           1,258 | T         |
    | 8,934 | srv5-dns                       | 2018-03-04 20:00:04 | B    | F     |         4 |           1,258 | T         |
    +-------+--------------------------------+---------------------+------+-------+-----------+-----------------+-----------+
    

    5) Which files were included in a backup job? Lists all paths and files included in the backup job (the ID is from the above command):

    *list files jobid=8934
    +----------+
    | Filename |
    +----------+
    | /var/lib/named/ |
    | /var/lib/named/root.cache |
    | /var/lib/named/sec |
    | /var/lib/named/sec/example.com.db |
    | /var/lib/named/sec/example2.net.db |
    | /var/lib/named/pri |
    +----------+
    +-------+----------+---------------------+------+-------+----------+----------+-----------+
    | JobId | Name     | StartTime           | Type | Level | JobFiles | JobBytes | JobStatus |
    +-------+----------+---------------------+------+-------+----------+----------+-----------+
    | 8,934 | srv5-dns | 2018-03-04 20:00:20 | B    | F     |        6 |    1,851 | T         |
    +-------+----------+---------------------+------+-------+----------+----------+-----------+
    

    6) Get the scheduled jobs. Which jobs will be executed and when:

    *status dir
    Scheduled Jobs:
    Level          Type     Pri  Scheduled          Job Name           Volume
    ===================================================================================
    Incremental    Backup    10  06-Mar-18 20:00    srv1-dns           *unknown*
    Incremental    Backup    10  06-Mar-18 20:00    srv2-dns           *unknown*
    Incremental    Backup    10  06-Mar-18 20:00    srv3-dns           *unknown*
    Incremental    Backup    10  06-Mar-18 20:00    srv4-dns           *unknown*
    Incremental    Backup    10  06-Mar-18 20:00    srv5-dns           *unknown*
    ...................................................................................
    Incremental    Backup    10  06-Mar-18 23:05    srv1-media         *unknown*
    Full           Backup    10  06-Mar-18 23:05    srv1-images        *unknown*
    ====
    Running Jobs:
    Console connected at 06-Mar-18 13:36
    No Jobs running.
    ====
    Terminated Jobs:
     JobId  Level    Files      Bytes   Status   Finished        Name
    ====================================================================
      9006  Incr        472    296.8 M  OK       05-Mar-18 23:05 srv1-dns
      9007  Incr     10,547    194.8 M  Error    05-Mar-18 23:05 srv2-dns
      9002  Incr         37    133.0 M  OK       05-Mar-18 23:05 srv3-dns
      8995  Incr         57    372.2 M  OK       05-Mar-18 23:06 srv4-dns
      9000  Incr        391    1.195 G  OK       05-Mar-18 23:07 srv5-dns
      9008  Full        832    7.139 G  OK       05-Mar-18 23:49 srv1-images
      9009  Full          1    1.493 G  OK       05-Mar-18 23:50 srv1-media
      9011  Full    315,027    121.6 G  OK       06-Mar-18 03:44 srv2-images
      9012  Full    314,804    93.85 G  OK       06-Mar-18 04:18 srv2-media
    ====
    *
    
  • Storages

  • Where are the backup files on your system? Trace the bacula media devices to the real path of your backup files.

    *status storage
    Automatically selected Storage: bind
    Connecting to Storage daemon bind at 192.168.0.10:9103
    srvbkp-sd Version: 7.0.5 (28 July 2014) x86_64-pc-linux-gnu ubuntu 16.04
    Daemon started 06-Nov-17 17:25. Jobs: run=5025, running=0.
     Heap: heap=135,168 smbytes=2,231,640 max_bytes=5,264,027 bufs=439 max_bufs=2,152
     Sizes: boffset_t=8 size_t=8 int32_t=4 int64_t=8 mode=0,0
    Running Jobs:
    No Jobs running.
    ====
    Jobs waiting to reserve a drive:
    ====
    Terminated Jobs:
     JobId  Level    Files      Bytes   Status   Finished        Name
    ===================================================================
      9071  Incr          0         0   OK       06-Mar-18 20:00 srv1-dns
      9074  Incr          0         0   OK       06-Mar-18 20:00 srv2-dns
      9073  Incr          0         0   OK       06-Mar-18 20:00 srv3-dns
      9075  Incr          5    2.043 K  OK       06-Mar-18 20:00 srv4-dns
      9078  Incr          5    2.042 K  OK       06-Mar-18 20:00 srv5-dns
      9077  Incr          0         0   OK       06-Mar-18 20:00 srv1-media
      9079  Incr          0         0   OK       06-Mar-18 20:00 srv1-images
      9076  Incr          0         0   OK       06-Mar-18 20:00 srv2-images
      9057  Full          0         0   Other    06-Mar-18 20:31 srv2-media
    ====
    Device status:
    Device "localstorage" (/mnt/storage1/bacula-storage/local) is not open.
    ==
    Device "media" (/mnt/storage1/bacula-storage/media) is not open.
    ==
    Device "bind" (/mnt/storage1/bacula-storage/bind) is not open.
    ==
    Device "image" (/mnt/storage1/bacula-storage/image) is not open.
    ==
    ====
    Used Volume status:
    ====
    Attr spooling: 0 active jobs, 34,753 bytes; 120 total jobs, 34,753 max bytes.
    ====
    *
    
  • Clients

    1) Show all client names in the bacula system, this could be useful to link a client name with a server and then to use the name in the next command (below in 2):

    *show client
    Client: name=srvbkp-fd address=192.168.0.5 FDport=9102 MaxJobs=1
          JobRetention=3 months  FileRetention=2 months  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    Client: name=srv1-dns address=192.168.0.100 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    Client: name=srv2-dns address=192.168.0.101 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    Client: name=srv3-dns address=192.168.0.103 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    Client: name=srv4-dns address=192.168.0.104 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    Client: name=srv5-dns address=192.168.0.105 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    Client: name=srv1-media address=192.168.0.106 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    Client: name=srv1-images address=192.168.0.107 FDport=9102 MaxJobs=1
          JobRetention=1 month  FileRetention=1 month  AutoPrune=1
      --> Catalog: name=allbackup address=localhost DBport=0 db_name=bacula
          db_driver=*None* db_user=bacula MutliDBConn=0
    *
    

    2) Show status of a client. We can use this command to check what is going on with the client at the moment of issuing the command, the last terminated backup jobs and their status. In addition we can check the connection between the Director daemon and the client daemon, because the Director connects at the moment we issue the command, so it is useful for debugging purposes:

    *status client=srv1-dns
    Connecting to Client srv1-dns at 192.168.0.100:9102
    
    srv1-dns-fd Version: 7.0.5 (28 July 2014)  x86_64-pc-linux-gnu ubuntu 16.04
    Daemon started 23-Feb-18 00:43. Jobs: run=8 running=0.
     Heap: heap=98,304 smbytes=188,701 max_bytes=571,361 bufs=64 max_bufs=97
     Sizes: boffset_t=8 size_t=8 debug=0 trace=0 mode=0,0 bwlimit=0kB/s
     Plugin: bpipe-fd.so 
    
    Running Jobs:
    Director connected at: 06-Mar-18 22:51
    No Jobs running.
    ====
    
    Terminated Jobs:
     JobId  Level    Files      Bytes   Status   Finished        Name 
    ===================================================================
      1832  Full          4    1.333 K  OK       01-Mar-18 23:48 srv1-dns
      1836  Incr          0         0   OK       02-Mar-18 00:01 srv1-dns
      1864  Incr          0         0   OK       02-Mar-18 20:30 srv1-dns
      1907  Incr          0         0   OK       03-Mar-18 20:00 srv1-dns
      1950  Full          4    1.333 K  OK       04-Mar-18 20:01 srv1-dns
      1994  Incr          0         0   OK       05-Mar-18 20:00 srv1-dns
      1037  Incr          0         0   OK       06-Mar-18 20:00 srv1-dns
    ====
    *
    
  • Filesets

    Which files will be included or excluded from the backup process. The lines starting with “I” mean “include”, the lines starting with “E” mean exclude.

     *show fileset
    FileSet: name=Full Set
     O M
     N
     I /usr/sbin
     N
     E /var/lib/bacula
     E /proc
     E /tmp
     E /sys
     E /.journal
     E /.fsck
     N
    FileSet: name=Catalog
     O M
     N
     I /var/lib/bacula/bacula.sql
     N
    FileSet: name=images
     O MfZ6
     N
     I /
     N
     E /proc
     E /tmp
     E /run
     E /dev
     E /sys
     N
    FileSet: name=bind
     O MZ6
     N
     I /var/lib/named
     N
    FileSet: name=media
     O MfZ6
     N
     I /
     N
     E /proc
     E /tmp
     E /run
     E /dev
     E /sys
     N
    
  • Schedule

    1) The all timeline plans for taking backups

    *show schedule
    Schedule: name=WeeklyCycle
      --> Run Level=Full
          hour=23 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=0 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=5
      --> Run Level=Differential
          hour=23 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=1 2 3 4 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=5
      --> Run Level=Incremental
          hour=23 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=1 2 3 4 5 6 
          wom=0 1 2 3 4 5 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=5
    Schedule: name=WeeklyCycleAfterBackup
      --> Run Level=Full
          hour=23 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 1 2 3 4 5 6 
          wom=0 1 2 3 4 5 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=10
    Schedule: name=images
      --> Run Level=Full
          hour=10 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=0 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Differential
          hour=10 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=1 2 3 4 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Incremental
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=1 2 3 4 5 6 
          wom=0 1 2 3 4 5 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
    

    2) The schedule plan for one client or groups of clients (server/s):

    *show schedule=bind
    Schedule: name=bind
      --> Run Level=Full
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=0 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Differential
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=0 
          wom=1 2 3 4 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
      --> Run Level=Incremental
          hour=20 
          mday=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 
          month=0 1 2 3 4 5 6 7 8 9 10 11 
          wday=1 2 3 4 5 6 
          wom=0 1 2 3 4 5 
          woy=0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 
          mins=0
    
  • Director

    1) Show the last messages of the backup processes. As you can see there is an error in one of the jobs, this error means that the client did not connect to the backup daemon (probably the same server with the Director), the problem was the firewall did not allow connections from this client (IP):

    *message
    06-Mar 20:00 srvbkp-dir JobId 1075: Start Backup JobId 1075, Job=srv1-bind.2018-03-06_20.00.01_05
    06-Mar 20:00 srvbkp-dir JobId 1075: Using Device "bind" to write.
    06-Mar 20:00 srvbkp-sd JobId 1066: Elapsed time=00:00:10, Transfer rate=0  Bytes/second
    06-Mar 20:00 srvbkp-dir JobId 1066: Bacula srvbkp-dir 7.0.5 (28Jul14):
      Build OS:               x86_64-pc-linux-gnu ubuntu 16.04
      JobId:                  1066
      Job:                    srv1-media.2018-03-06_20.00.00_56
      Backup Level:           Incremental, since=2018-03-06 20:00:14
      Client:                 "srv1-media" 7.0.5 (28Jul14) x86_64-pc-linux-gnu,ubuntu,16.04
      FileSet:                "bind" 2017-11-07 17:19:45
      Pool:                   "bind-incr" (From Job IncPool override)
      Catalog:                "allbackup" (From Client resource)
      Storage:                "bind" (From Job resource)
      Scheduled time:         06-Mar-2018 20:00:00
      Start time:             06-Mar-2018 20:00:15
      End time:               06-Mar-2018 20:00:26
      Elapsed time:           11 secs
      Priority:               10
      FD Files Written:       0
      SD Files Written:       0
      FD Bytes Written:       0 (0 B)
      SD Bytes Written:       0 (0 B)
      Rate:                   0.0 KB/s
      Software Compression:   None
      VSS:                    no
      Encryption:             no
      Accurate:               no
      Volume name(s):        
      Volume Session Id:      1025
      Volume Session Time:    1509989534
      Last Volume Bytes:      4,618 (4.618 KB)
      Non-fatal FD errors:    0
      SD Errors:              0
      FD termination status:  OK
      SD termination status:  OK
      Termination:            Backup OK
    06-Mar 20:01 srvbkp-dir JobId 1057: Using Device "bind" to write.
    06-Mar 20:05 srv2-dns-fd JobId 1057: Warning: bsock.c:112 Could not connect to Storage daemon on 192.168.0.5:9103. ERR=Connection timed out
    Retrying ...
    06-Mar 20:31 srv2-dns-fd JobId 1057: Fatal error: bsock.c:118 Unable to connect to Storage daemon on 192.168.0.5:9103. ERR=Interrupted system call
    06-Mar 20:31 srv2-dns-fd JobId 1057: Fatal error: job.c:1893 Failed to connect to Storage daemon: 192.168.0.5:9103
    06-Mar 20:31 srvbkp-dir JobId 1057: Fatal error: Bad response to Storage command: wanted 2000 OK storage
    , got 2902 Bad storage
    06-Mar 20:31 srvbkp-dir JobId 1057: Error: Bacula srvbkp-dir 7.0.5 (28Jul14):
      Build OS:               x86_64-pc-linux-gnu ubuntu 16.04
      JobId:                  1057
      Job:                    srv2-dns.2018-03-06_20.00.00_47
      Backup Level:           Full (upgraded from Incremental)
      Client:                 "srv2-dns" 7.0.5 (28Jul14) x86_64-pc-linux-gnu,ubuntu,16.04
      FileSet:                "bind" 2017-11-07 17:19:45
      Pool:                   "bind-full" (From Job FullPool override)
      Catalog:                "allbackup" (From Client resource)
      Storage:                "bind" (From Job resource)
      Scheduled time:         06-Mar-2018 20:00:00
      Start time:             06-Mar-2018 20:00:00
      End time:               06-Mar-2018 20:31:22
      Elapsed time:           31 mins 22 secs
      Priority:               10
      FD Files Written:       0
      SD Files Written:       0
      FD Bytes Written:       0 (0 B)
      SD Bytes Written:       0 (0 B)
      Rate:                   0.0 KB/s
      Software Compression:   None
      VSS:                    no
      Encryption:             no
      Accurate:               no
      Volume name(s):        
      Volume Session Id:      1201
      Volume Session Time:    1509989534
      Last Volume Bytes:      1 (1 B)
      Non-fatal FD errors:    2
      SD Errors:              0
      FD termination status:  Error
      SD termination status:  Waiting on FD
      Termination:            *** Backup Error ***
    

    2) reload – reload the configuration files of bacula system. The daemons will re-read all configuration files in “/etc/bacula”. Unfortunately there is no output:

    *reload
    *
    

bacula fatal error – Unable to connect to Storage daemon

Bacula is an open software enterprise backup system! Check out the official site here
Complex but useful software, which could automate the whole backup process of all your servers.
Some errors are easy to track some are not, so here is one error with a misleading error message if you do not know or forget the details of how the daemons works.

Here is the error extracted from the logs:

01-Sep 00:45 backup01-de-dir JobId 8789: No prior Full backup Job record found.
01-Sep 00:45 backup01-de-dir JobId 8789: No prior or suitable Full backup found in catalog. Doing FULL backup.
01-Sep 00:45 backup01-de-dir JobId 8789: Job srv123us.2017-09-01_00.45.28_34 waiting 103 seconds for scheduled start time.
01-Sep 00:47 backup01-de-dir JobId 8789: Start Backup JobId 8789, Job=srv123us.2017-09-01_00.45.28_34
01-Sep 00:47 backup01-de-dir JobId 8789: Using Device "web" to write.
01-Sep 00:51 srv123us-fd JobId 8789: Warning: bsock.c:112 Could not connect to Storage daemon on 1.1.1.1:9103. ERR=Connection timed out
01-Sep 01:17 srv123us-fd JobId 8789: Fatal error: bsock.c:118 Unable to connect to Storage daemon on 1.1.1.1:9103. ERR=Interrupted system call
01-Sep 01:17 srv123us-fd JobId 8789: Fatal error: job.c:1893 Failed to connect to Storage daemon: 1.1.1.1:9103
01-Sep 01:17 backup01-de-dir JobId 8789: Fatal error: Bad response to Storage command: wanted 2000 OK storage
01-Sep 01:17 backup01-de-dir JobId 8789: Error: Bacula backup01-de-dir 7.0.5 (28Jul14):
 Build OS:               x86_64-pc-linux-gnu ubuntu 16.04
  JobId:                  8789
  Job:                    srv123us.2017-09-01_00.45.28_34
  Backup Level:           Full (upgraded from Incremental)
  Client:                 "srv123us" 7.0.5 (28Jul14) x86_64-pc-linux-gnu,ubuntu,16.04
  FileSet:                "web" 2017-11-07 17:19:45
  Pool:                   "web-full" (From Job FullPool override)
  Catalog:                "ucdn" (From Client resource)
  Storage:                "web" (From Job resource)
  Scheduled time:         01-Sep-2018 00:47:11
  Start time:             01-Sep-2018 00:47:11
  End time:               01-Sep-2018 01:17:23
  Elapsed time:           30 mins 12 secs
  Priority:               10
  FD Files Written:       0
  SD Files Written:       0
  FD Bytes Written:       0 (0 B)
  SD Bytes Written:       0 (0 B)
  Rate:                   0.0 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               no
  Volume name(s):         
  Volume Session Id:      4719
  Volume Session Time:    1510075534
  Last Volume Bytes:      0 (0 B)
  Non-fatal FD errors:    2
  SD Errors:              0
  FD termination status:  Error
  SD termination status:  Waiting on FD
  Termination:            *** Backup Error ***

But when we check the status of client from “bconsole” (Bacula’s management Console), everything seems OK, the backup server (Director daemon = bacula-dir) connects and get the report from the client daemon (Bacula File service = bacula-fd) in the server, even when you run a backup job, the status report is OK, the backup is running on the client, here is the output:

srv@local ~ # bconsole
Connecting to Director localhost:9101
1000 OK: 1 backup01-de-dir Version: 7.0.5 (28 July 2014)
Enter a period to cancel a command.
*status
Status available for:
     1: Director
     2: Storage
     3: Client
     4: Scheduled
     5: All
Select daemon type for status (1-5): 3
The defined Client resources are:
     1: srv1us
     2: srv2us
     3: srv123us
Select Client (File daemon) resource (1-3): 3
Connecting to Client srv123us at 108.61.250.36:9102
srv123us-fd Version: 7.0.5 (28 July 2014)  x86_64-pc-linux-gnu ubuntu 16.04
Daemon started 23-Feb-17 00:43. Jobs: run=1 running=0.
 Heap: heap=98,304 smbytes=571,344 max_bytes=571,361 bufs=97 max_bufs=97
 Sizes: boffset_t=8 size_t=8 debug=0 trace=0 mode=0,0 bwlimit=0kB/s
 Plugin: bpipe-fd.so 

Running Jobs:
JobId 8789 Job srv123us.2017-09-01_00.45.28_34 is running.
    Incremental Backup Job started: 01-Sep-17 00:45
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0 Errors=0
    Bwlimit=0
    Files: Examined=5 Backed up=0
    SDReadSeqNo=6 fd=5
Director connected at: 01-Sep-17 01:10
====

Terminated Jobs:
====

As you can see, everything seems OK of the status, there was a running job in the client server and it seemed the backup process had been running without errors for more then 20 minutes, but then suddenly got Fatal error (the first log):

01-Sep 00:51 srv123us-fd JobId 8789: Warning: bsock.c:112 Could not connect to Storage daemon on 1.1.1.1:9103. ERR=Connection timed out
01-Sep 01:17 srv123us-fd JobId 8789: Fatal error: bsock.c:118 Unable to connect to Storage daemon on 1.1.1.1:9103. ERR=Interrupted system call
01-Sep 01:17 srv123us-fd JobId 8789: Fatal error: job.c:1893 Failed to connect to Storage daemon: 1.1.1.1:9103
01-Sep 01:17 backup01-de-dir JobId 8789: Fatal error: Bad response to Storage command: wanted 2000 OK storage

And the problem is that, the Director (backup server) connects to the File Service of the client (the daemon on the client), but the opposite connection is not possible! When the backup is ready, the client daemon bacula file service connects to the bacula storage service (which could be on the same server with the director, but it could be on another server) to send the backup files and here is the problem! Client could not connect to the storage! So always check the two way connections: backup server -> client server-port:9102 and backup server-port:9103 (or storage server) <- client.
In the world of bacula:

bacula-dir -> bacula-fd:9102

bacula-sd:9103 -> bacula-fd

Misleading error on causal look it seems like bacula-sd is returning error to bacula-fd (which would mean that bacula-fd could connect to bacula-sd after all), but in reality bacula-dir received and logged that bacula-fd did not connect to bacula-sd resulting in Fatal error.

In our situation the firewall of the backup server was denying the connections from the client, but it could be a DNS resolve issue or another network problem. Most common problems are firewall or DNS resolve issues. The solution – just add accept rule for the IP of the client to connect to port 9103 of the backup (storage) server.