The following post appears to be older than 100 days. I therefore cannot guarantee that any technical information in this post is still valid today.

Please consider to also look for other, more up to date resources!
2007/11/22

Parsing Simple Config Files In Bash

Bash (or insert any other shell you prefer over bash) scripts are the Swiss army knife of every system administrator when it comes to solve certain tasks like backups or other maintenance related things on a Linux machine. Usually such scripts are simple and there's often no need to use configuration files. If you still need to rely on configuration files for different scenarios you can just use one bash script to set certain variables for each case and then source it in the main script at a certain point. However, I am currently working on a little bit more sophisticated backup script and I decided to go another way and make use of a configuration file which holds a set of options for each backup target (targets like system, home of user foo, media and so forth). Of course I could have chosen to use one configuration file for every single backup target but I wanted to have everything in one place (and I was in for a little challenge to eventually gain better bash scripting skills ;-)). The layout of the configuration file had to be simple - I finally decided to use the following.

The meaning of the options isn't of any interest - this is just an example to proof the point.

The following is a simple configuration file:

# default settings
default {
DATE_PREFIX=$(date -I)
EXT_FULL="full"
EXT_DIFF="diff"
SSHFS_OPTS="-C"
DAR_OPTS="-v -m 256 -y -s 600M -D"
DAR_NOCOMPR="-Z '*.gz' -Z '*.bz2' -Z '*.zip' -Z '*.png'"
}

# backup target system
system {
SRC_DIR="/"
DEST_DIR="/mnt/data/backups/tatooine"
PREFIX="system"
TYPE="R"
HOST="chi@coruscant"
}

# backup target home
home {
SRC_DIR="/home/user"
DEST_DIR="/mnt/data/backups/tatooine"
PREFIX="home-nomedia"
TYPE="R"
HOST="chi@coruscant"
DAR_EXCLUDES="media"
}

You can think of the option assignments just like when your assigning values to a bash variable (in fact it's more or less just that - as you can see at the DATE_PREFIX option you can even use bash expressions).

The following bash function parses the above configuration file. It takes exactly one argument which has to be the configuration section you want to get the options for. It scans the configuration file for the right section, if it finds it, each line of the section is eval'd to finally assign the variables which then can be used later in the script.

At the moment the function doesn't support in-line comments in configuration files and it's not able able to detect configuration files with broken syntax like a missing ”}” for example.
parseconfig.sh
#!/usr/bin/env bash
# @author Michael Klier <chi@chimeric.de>
 
function readconf() {
 
    match=0
 
    while read line; do
        # skip comments
        [[ ${line:0:1} == "#" ]] && continue
 
        # skip empty lines
        [[ -z "$line" ]] && continue
 
        # still no match? lets check again
        if [ $match == 0 ]; then
 
            # do we have an opening tag ?
            if [[ ${line:$((${#line}-1))} == "{" ]]; then
 
                # strip "{"
                group=${line:0:$((${#line}-1))}
                # strip whitespace
                group=${group// /}
 
                # do we have a match ?
                if [[ "$group" == "$1" ]]; then
                    match=1
                    continue
                fi
 
                continue
            fi
 
        # found closing tag after config was read - exit loop
        elif [[ ${line:0} == "}" && $match == 1 ]]; then
            break
 
        # got a config line eval it
        else
            eval $line
        fi
 
    done < "$CONFIG"
}

The function uses some of the more advanced bash features like parameter substitution a.s.o. which I won't explain here. For a good read on the whole bash scripting topic I recommend the Advanced Bash Scripting Guide. It was my main resource for the above function as well ;-).

Using the above function is quite easy. Here's a simple example:

parseconfig.sh
# function definition comes before this line
 
CONFIG="/home/user/.sampleconfig"
 
readconf "default"
 
echo $DATE_PREFIX
echo $DAR_OPTS
echo $DAR_NOCOMPR

And here goes the output:

% ./parseconfig.sh 
2007-11-22
-v -m 256 -y -s 600M -D
-Z '*.gz' -Z '*.bz2' -Z '*.zip' -Z '*.png'

That's it :-). As I said before there are other/easier ways of feeding scripts with configuration options. This was just for my personal exercise and pleasure ;-).

Happy scripting!

Update: I Just found out that the function contained 3 lines of completely obsolete code and removed them ;-).

Comments

1

Stripping inline comments should be easy by throwing in a few sed calls. But I'm a amazed by what you achieved so far without any call to grep, awk and sed!

2007/11/23 00:10
2

:-), well in fact the title should read “Parsing Simple Config Files In Pure Bash” because it was my goal not to use other helper programs like sed, awk, grep and so forth ;-).

2007/11/23 00:16
3

Whew, it must have been some serious brain-twisting to geht the parsing right in pure bash.

But somehow this seems not to be very KISS-y… ;-)

2007/11/23 07:50
4

Hehe, no you're right - it's definitely not very KISS ;-).

2007/11/23 10:05
5

If you're going to use bash semantics in your script, please use the #!/bin/bash sebang as sh may not be bash on many systems. Although #!/usr/bin/env bash maybe more portable as bash could be in say /usr/local/bin instead!

2008/01/07 13:51
6

Hi roy, you're right of course, using sh developed to be a bad habit of mine ;-), thanks for the hint!

2008/01/07 16:06
7

Simple and useful, thanks. Here's a bit improvement. It's good to have section common for all profiles. Keyword 'function' is redundant.

4,5c4,5
< function readconf() {
<  
---
> readconf() {
> 
16c16
<         if [ $match == 0 ]; then
---
>         if [[ $match == 0 ]]; then
27c27
<                 if [[ "$group" == "$1" ]]; then
---
>                 if [[ "$group" == "$1" || "$group" == '*' ]]; then
37c37,38
<             break
---
>             match=0
>             continue
aidecoe
2008/05/18 12:26
8

Hi aidecoe,

thanks for your comment. Having a common section for all profiles is indeed a useful addition, though you could also use a section group named default instead of *.

PS.: You can use wiki syntax in the comments ;-).

2008/05/18 14:08
9

Yes, but only default group is parsed. Patch above always parses '*' and the one given by argument. Btw. '*' is not good in this case, I just noticed. :) Bash expands it to all files in current directory. Better use some literal.

PS. Thanks for correction.

aidecoe
2008/05/18 22:36
10

Nice example, love the idea of having different configuration blocks, below is an example regexp which allows comments with semicolon and rhomb sign. Blank or space filled lines are ignored, inline comments can be used also ;)

Maybe you could try to merge that into your code.

if ! [[ $LINE =~ ^[\ ]{0,}[\#\;] ]] ;then
    if ! [[ $LINE =~ ^$ ]] ;then
        eval "$LINE"
    fi
fi
2008/07/12 20:13
11

This works really nice in Tomcat to setup different servers in a cluster and have a centralized server configurations.

thanks!

enthusiast
2009/08/13 20:12



JKNKH