Project

General

Profile

Produkční server warden.cesnet.cz

Zpět na obsah dokumentu

 Hostname: warden.cesnet.cz

 OS:       Debian Squeeze
           Linux warden 2.6.32-5-amd64 #1 SMP Mon Oct 3 03:59:20 UTC 2011 x86_64 GNU/Linux

 CPU:      Intel XEON(tm) L5640 processor @2.26GHz

 RAM:                   total       used       free     shared    buffers     cached
           Mem:      33003928     264744   32739184          0       8420     124096
           -/+ buffers/cache:     132228   32871700
           Swap:      1952760          0    1952760

 HDD:
           Filesystem            Size  Used Avail Use% Mounted on
           /dev/sda1             9.2G  525M  8.2G   6% /
           tmpfs                  16G     0   16G   0% /lib/init/rw
           udev                   16G  160K   16G   1% /dev
           tmpfs                  16G     0   16G   0% /dev/shm
           /dev/sda3             538G  340M  511G   1% /var

  RAID: 
           hardwarový 5

  KONFIGURACE:
           Dell PowerEdge R410 Rack Chassis for Up to 4x 3.5" Hot Plug HDDs, LCD diagnostics
           2 x Intel XEON L5640 processor (2.26GHz, 6C, 12M Smart Cache), DDR3-800/1066/1333 MHz
           32GB Memory for 2CPU (8x4GB Dual Rank UDIMMs) 1333MHz
           3 x 300GB SAS 6Gbps 15k 3.5" HD Hot Plug
           PERC H700A RAID Controller, 1GB NV Cache, For Hot Plug HDD Chassis
           16X DVD+/-RW SATA Drive with SATA Cable

Nasazovani systemu Warden

Verze Datum (od) Cas (od) Nasadil
2.0 ? ? Tomas Plesnik, Pavel Kacha, Honza Mach
2.1 26.2.2013 09:41:02 Tomas Plesnik, Pavel Kacha, Honza Mach

Konfigurace serveru

Nastavení monitorování systému

Pomocí systému Nagios a nástrojů Logwatch a Apticron:

Instalace balíčků:

aptitude install nagios-nrpe-server nagios-plugins-basic logwatch apticron host

Konfigurace v /etc/nagios/nrpe.cfg:

#nagios.cesnet.cz pool
allowed_hosts=195.113.187.74,195.113.187.75,195.113.187.76,195.113.187.77,195.113.187.78

# watched services
command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib/nagios/plugins/check_load -w 45,40,20 -c 50,50,40
command[check_disk_/]=/usr/lib/nagios/plugins/check_disk -w 20% -c 10% -p /
command[check_disk_/var]=/usr/lib/nagios/plugins/check_disk -w 20% -c 10% -p /var
command[check_zombie_procs]=/usr/lib/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 350 -c 500
command[check_ssh]=/usr/lib/nagios/plugins/check_ssh -H localhost
command[check_ntp_time]=/usr/lib/nagios/plugins/check_ntp_time -H tik.cesnet.cz -w 0.5 -c 1
command[check_warden]=/usr/lib/nagios/plugins/check_procs -c 1:1 -a /opt/warden-server/bin/warden-server.pl

Nastavení zálohování

O zálohování se stará skript /root/bin/backup, který je spouštěn cronem každý den ve 22:30 hodin. Skript je z důvodu efektivity nastaven tak, aby každou neděli provedl plnou zálohu a každý další den tvořil pouze inkrementální rozdílové zálohy. Zálohování se provádí na server neant.cesnet.cz.

Nastavení logování

Konfigurace v /etc/syslog-ng/syslog-ng.conf:

options { long_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no);
          owner("root"); group("adm"); perm(0640); log_fifo_size(10000); mark_freq(300); stats_freq(600);
          stats_level(1); time_reopen(10); bad_hostname("^gconfd$");
};

destination d_warden { file("/var/log/warden.log" group("warden")); };
destination d_net { tcp("vinovago.cesnet.cz" 
                        port(514)
                        tls(
                                key_file("/etc/ssl/private/warden.cesnet.cz.key")
                                cert_file("/etc/ssl/certs/warden.cesnet.cz.pem")
                                ca_dir("/etc/ssl/trusted_ca")
                                trusted_dn("CN=vinovago.cesnet.cz, O=CESNET, C=CZ")
                        )
                        flush_lines(100)
                        flush_timeout(5000)
                     );  
                  };   

filter f_warden { facility(local7); };

log { source(s_src); filter(f_warden); destination(d_warden); };
log { source(s_src); destination(d_net); };

Konfigurace v /etc/logrotate.d/warden:

/var/log/warden.log {
   rotate 12
   missingok
   notifempty
   weekly
   compress
   delaycompress
   dateext
}

SSL certifikáty

serverovy certifikat:     /etc/ssl/certs/warden.cesnet.cz.pem
klic:            /etc/ssl/private/warden.cesnet.cz.key
CA certifikat budle:    /etc/ssl/certs/tcs-ca-bundle.pem
TERENA SSL CA:        /etc/ssl/certs/TERENA_SSL_CA.pem

(Budle CA obsahuje chain vsech CA az ke koreni)
# Get the certificate hash
openssl x509 -noout -hash -in /etc/ssl/certs/TERENA_SSL_CA.pem

# Install the symbolic link
ln -s /etc/ssl/certs/TERENA_SSL_CA.pem /etc/ssl/certs/9df51c42.0

# Verify the installation
openssl verify /etc/ssl/certs/mentat.cesnet.cz.pem

Instalovane baliky

Provozni

aptitude install ntp mc joe vim screen htop apt-show-versions less host git

Vyvojove

apt-get install libsoap-lite-perl libfile-pid-perl sqlite3 libdbd-sqlite3-perl libformat-human-bytes-perl libnet-cidr-lite-perl libdatetime-perl
  • libsoap-lite-perl
    The following extra packages will be installed:
      libclass-inspector-perl libconvert-binhex-perl libcrypt-ssleay-perl libfcgi-perl libfont-afm-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libio-socket-ssl-perl libio-stringy-perl
      libmailtools-perl libmime-tools-perl libnet-libidn-perl libnet-ssleay-perl libossp-uuid-perl libossp-uuid16 libtask-weaken-perl libtimedate-perl liburi-perl libwww-perl libxml-parser-perl
    Suggested packages:
      libdata-dump-perl libio-socket-inet6-perl uuid libapache2-mod-perl2 libnet-jabber-perl libmime-lite-perl
    The following NEW packages will be installed:
      libclass-inspector-perl libconvert-binhex-perl libcrypt-ssleay-perl libfcgi-perl libfont-afm-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libio-socket-ssl-perl libio-stringy-perl
      libmailtools-perl libmime-tools-perl libnet-libidn-perl libnet-ssleay-perl libossp-uuid-perl libossp-uuid16 libsoap-lite-perl libtask-weaken-perl libtimedate-perl liburi-perl libwww-perl libxml-parser-perl
    
  • libfile-pid-perl
    The following extra packages will be installed:
      libclass-accessor-perl libsub-name-perl
    The following NEW packages will be installed:
      libclass-accessor-perl libfile-pid-perl libsub-name-perl
    
  • sqlite3 (3.7.3.)
    Suggested packages:
      sqlite3-doc
    The following NEW packages will be installed:
      sqlite3
    
  • libdbd-sqlite3-perl
    The following extra packages will be installed:
      libdbi-perl libnet-daemon-perl libplrpc-perl
    The following NEW packages will be installed:
      libdbd-sqlite3-perl libdbi-perl libnet-daemon-perl libplrpc-perl
    
  • libformat-human-bytes-perl
  • libnet-cidr-lite-perl
  • libdatetime-perl
    The following extra packages will be installed:
      libclass-singleton-perl libdatetime-locale-perl libdatetime-timezone-perl liblist-moreutils-perl libparams-validate-perl
    The following NEW packages will be installed:
      libclass-singleton-perl libdatetime-locale-perl libdatetime-perl libdatetime-timezone-perl liblist-moreutils-perl libparams-validate-perl
    

Správa klientů Wardenu

Registrace odesilajiciho klienta

# sudo /opt/warden-server/bin/registerSender.pl -n <hostname> -r <requestor> -s <service_name> -d <description_tags> -i <IP or subnet>
# mirrorClientsToSandbox.sh

Registrace prijimajiciho klienta

# sudo /opt/warden-server/bin/registerReceiver.pl -n <hostname> -r <requestor> -t <events_type> [-o] -i <IP or subnet>
# mirrorClientsToSandbox.sh

mirrorClientsToSandbox.sh zrcadlí databázi klientů z warden.cesnet.cz na warden-sandbox.cesnet.cz (beze změny klientů v horní půlce namespace client_id na warden-sandbox, které je určené pro čisté hraní bez návaznosti na ostrý server).
Zpět na obsah dokumentu