Misc SSH as a VPN links

Just collecting some links that look interesting. My plan is to use SSH as a VPN server and I'll need to figure out how Mac and Windows clients can connect in.

Using a Linux L2TP/IPsec VPN server

arstechnica - teach me about ssh, vpn, allow remote connections

about.com - VPN Setup

Q&A on using the VPN service (www.doc.ic.ac.uk/csg/faqs/)

Gentoo: emerge samba fails while compiling rpctorture.c

Got the following error while trying to emerge samba into my Gentoo box.

Compiling torture/rpctorture.c
make: *** Waiting for unfinished jobs....
torture/rpctorture.c:27: error: `global_myname' redeclared as different kind of symbol
include/proto.h:1019: error: previous declaration of `global_myname'
torture/rpctorture.c:57: warning: `struct client_info' declared inside parameter list
torture/rpctorture.c:57: warning: its scope is only this definition or declaration, which is probably not what you want
torture/rpctorture.c: In function `rpcclient_connect':
torture/rpctorture.c:62: error: dereferencing pointer to incomplete type
torture/rpctorture.c:62: error: dereferencing pointer to incomplete type
torture/rpctorture.c:63: error: dereferencing pointer to incomplete type
torture/rpctorture.c:66: error: dereferencing pointer to incomplete type
torture/rpctorture.c:66: error: dereferencing pointer to incomplete type
torture/rpctorture.c:68: error: dereferencing pointer to incomplete type
torture/rpctorture.c:68: error: dereferencing pointer to incomplete type
torture/rpctorture.c: At top level:
torture/rpctorture.c:90: warning: `struct client_info' declared inside parameter list
torture/rpctorture.c: In function `run_enums_test':
torture/rpctorture.c:96: warning: passing arg 1 of `rpcclient_connect' from incompatible pointer type
torture/rpctorture.c:102: error: dereferencing pointer to incomplete type
torture/rpctorture.c:102: error: dereferencing pointer to incomplete type
torture/rpctorture.c: At top level:
torture/rpctorture.c:134: warning: `struct client_info' declared inside parameter list
torture/rpctorture.c: In function `run_ntlogin_test':
torture/rpctorture.c:140: warning: passing arg 1 of `rpcclient_connect' from incompatible pointer type
torture/rpctorture.c:146: error: dereferencing pointer to incomplete type
torture/rpctorture.c:146: error: dereferencing pointer to incomplete type
torture/rpctorture.c: At top level:
torture/rpctorture.c:167: warning: `struct client_info' declared inside parameter list
torture/rpctorture.c: In function `main':
torture/rpctorture.c:233: error: storage size of `cli_info' isn't known
torture/rpctorture.c:377: error: `scope' undeclared (first use in this function)
torture/rpctorture.c:377: error: (Each undeclared identifier is reported only once
torture/rpctorture.c:377: error: for each function it appears in.)
torture/rpctorture.c:535: warning: passing arg 5 of `create_procs' from incompatible pointer type
torture/rpctorture.c:539: warning: passing arg 5 of `create_procs' from incompatible pointer type
make: *** [torture/rpctorture.o] Error 1
* rpctorture didn't build
running build
running build_py
running build_ext
--------------------------- ACCESS VIOLATION SUMMARY ---------------------------
LOG FILE = "/var/log/sandbox/sandbox-net-fs_-_samba-3.0.14a-r2-21241.log"

access_wr: /etc/krb5.conf
--------------------------------------------------------------------------------
#

Here are my current USE flags:

# cat /etc/make.conf

# These settings were set by the catalyst build script that automatically built this stage
# Please consult /etc/make.conf.example for a more detailed example
CFLAGS="-Os -mcpu=i686"
CHOST="i386-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j2"

GENTOO_MIRRORS="http://gentoo.osuosl.org/"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"


USE="apache2 kerberos ldap postgres samba -alsa -apm -arts -bitmap-fonts -gnome -gtk -gtk2 -kde -mad -mikmod -motif -opengl -oss -qt -quicktime -sdl -truetype -truetype-fonts -type1-fonts -X -xmms -xv"

# cat /etc/make.profile/make.defaults

# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/profiles/default-linux/x86/2005.1/make.defaults,v 1.4 2005/08/29 22:20:25 wolf31o2 Exp $

USE="alsa apm arts avi berkdb bitmap-fonts crypt cups eds emboss encode fortran foomaticdb gdbm gif gnome gpm gstreamer gtk gtk2 imlib ipv6 jpeg kde libg++ libwww mad mikmod motif mp3 mpeg ncurses nls ogg oggvorbis opengl oss pam pdflib perl png python qt quicktime readline sdl spell ssl tcpd truetype truetype-fonts type1-fonts vorbis X xml2 xmms xv zlib"
#

I'm still searching for a solution to this issue. I've heard it has to do with trying to use the kerberos USE flag (which is not an optional flag for me). The closest possible solution in Google is on the Gentoo forums (Problems upgrading to Samba 3.0.14a-r2!). The user, "jpnag", posts a solution.

The solution involves editing the ebuild file for Samba. This is where you will need to become a bit more knowledgeable about how portage and emerge works (see "man make.conf" for details on some of this along with "man portage").

By default, portage downloads and installs packages under the "/usr/portage/" tree (defined by "PORTDIR=" in your "/etc/make.conf" file or "/etc/make.profile/make.defaults" file). There is also an optional define, "PORTDIR_OVERLAY=", which you can use to point at a tree containing user-built ebuild files that are not updated by "emerge --sync". Essentially, the second tree will overlay the first. So if you have "package X" in both trees, only the one in the overlay tree will get compiled.

Now to create the backup copy of the broken Samba ebuild. If you have not already added "PORTDIR_OVERLAY=" to your "make.conf" file, you should also do this.

# cd /etc
etc # echo 'PORTDIR_OVERLAY="/usr/local/portage"' >> /etc/make.conf
etc # cd /usr/local
local # ls /usr/portage/net-fs/samba/
local # mkdir portage ; cd portage
portage # mkdir net-fs ; cd net-fs
net-fs # mkdir samba ; cd samba
samba # cp -a /usr/portage/net-fs/samba/* .
samba # ls -l samba-3.0.14a-r2.ebuild
samba # nano -w samba-3.0.14a-r2.ebuild

Now hit [Ctrl-W] and type "src_compile", which will take you straight to the following code block:

rc_compile() {
        ebegin "Running autoconf"
                autoconf
        eend $?

        local myconf
        local mymods
        local mylangs

        if use xml || use xml2 ;
        then   
                mymods="xml,${mymods}"
        fi

Somewhere towards the start of the funciton, add the line "addpredict /etc/krb5.conf".

src_compile() {
        ebegin "Running autoconf"
                autoconf
        eend $?

        local myconf
        local mymods
        local mylangs

        addpredict /etc/krb5.conf

        if use xml || use xml2 ;
        then
                mymods="xml,${mymods}"
        fi

Create the ebuild digest (MD5 signatures) for the patched package.

samba # ebuild /usr/local/portage/net-fs/samba/samba-3.0.14a-r2.ebuild digest
>>> Generating digest file...
<<< samba-3.0.14a.tar.gz
<<< samba-vscan-0.3.6.tar.bz2
<<< samba-3-gentoo-0.3.3.tar.bz2
>>> Generating manifest file...
<<< ChangeLog
<<< metadata.xml
<<< samba-3.0.14a-r2.ebuild
<<< samba-3.0.14a-r3.ebuild
<<< samba-3.0.20-r1.ebuild
<<< samba-3.0.20a.ebuild
<<< samba-3.0.20b.ebuild
<<< files/digest-samba-3.0.14a-r2
<<< files/README.gentoo
<<< files/digest-samba-3.0.14a-r3
<<< files/digest-samba-3.0.20-r1
<<< files/digest-samba-3.0.20a
<<< files/digest-samba-3.0.20b
>>> Computed message digests.

samba # emerge -pv samba

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild N ] net-fs/samba-3.0.14a-r2 -acl +cups -doc +kerberos +ldap -libclamav -mysql -oav +pam +postgres +python -quotas +readline (-selinux) -winbind -xml +xml2 0 kB [1]

Total size of downloads: 0 kB
Portage overlays:
[1] /usr/local/portage

samba # emerge samba

(crosses fingers)

Unix/Lunix Filesystem Hierarchy Standard

The Filesystem Hierarchy Standard is a set of guidelines designed to get various distrobutions of Linux/Unix all on the same page with regards to what goes where in the file system.

For us laymen, it's a decent guide to what is where in the filesystem (why is there a /sbin and a /usr/local/sbin?). It helps answer questions like, "If I have a backup script, where should I put the file?".

Gentoo: eth0 does not exist

So, oops. When I built my Celeron box, I didn't include the driver for my Netgear FA310TX Rev-D1 in the kernel build. The Universal CD automatically detects the network card properly, now I just need to get it configured into my kernel build (using "make menuconfig"). The Universal CD (2005.1) detected and configured it using the Tulip driver (Lite-On 82c168).

So I know it works, I just don't have it right and proper. My original build for this kernel is:

Gentoo 2005.1 Software RAID (part 2) Celeron CPU

During that build, I didn't touch network devices at all. It had automatically selected the RealTek network device driver under the EISA option.

# cd /usr/src/linux
# make menuconfig

(D)evice drivers
--> N(e)tworking support
--> --> N(e)twork device support (should already be BUILT-IN)
--> --> --> (E)thernet (10 or 100Mbit)
--> --> --> --> (T)ulip family network device support
--> --> --> --> --> "(T)ulip" family network device support (turn ON as BUILT-IN)
--> --> --> --> --> --> (D)ECchip Tulip (dc2114x) PCI support (turn ON as BUILT-IN)
--> --> --> --> --> --> --> (I left the sub-options alone)
--> --> --> --> (E)ISA, VLB, PCI and on board controllers (turn OFF)

Now, make sure that /boot is mounted, then compile your new kernel.

# make && make modules_install
# cp arch/i386/boot/bzImage /boot/kernel-2.6.12-Oct2005
# cp System.map /boot/System.map-2.6.12-Oct2005
# cp .config /boot/config-2.6.12-Oct2005
# nano -w /boot/grub/grub.conf

Contents of my grub.conf file:

# Which listing to boot as default. 0 is the first, 1 the second etc.
default 0
timeout 30

# Oct 2005 recompile kernel to add Netgear FA310TX rev D1
title=Gentoo Linux 2.6.12 (Oct 20 2005)
root (hd0,0)
kernel /kernel-2.6.12-Oct2005 root=/dev/md2

# Sep 2005 installation (software RAID, no LVM2)
title=Gentoo Linux 2.6.12 (Sep 22 2005)
root (hd0,0)
kernel /kernel-2.6.12-Sep2005 root=/dev/md2

Not technically difficult, if you can find out what device driver to use. Which, is why I try to document as much of this stuff as possible.

Imaging a Tecra 8200 Windows 2000 install using Knoppix, NTFSClone, and Samba

One of the more annoying things that our users do is to get their Windows workstations into an unworkable state after 9 months or so. We've tried user education, we use things like spyware removers, eliminating the use of Internet Explorer, etc, but there are a lot of times where it's less time and trouble for us to reset the machine back to a known point. Now, there are a few ways to do this such as paying for programs like Norton Ghost or Acronis True Image. Both of those work but they cost $50/seat roughly. But I'd like to do it for less. Or at least take a shot at doing it for less.

(I do use Acronis True Image on a few of my systems. It's good for making more frequent snapshots without having to reboot to a seperate operating system. It also has an incremental mode that can be used to keep a single image up to date. For user simplicity, it's hands-down the winner over Ghost.)

First off, you will need to download and burn the Knoppix Boot CD. I'm using version 4.0.2 which hopefully has what I need (see this Slashdot comment, "where is ntfsclone", from Oct 2004). The version that shows up on the 4.0.2 Knoppix CD is "ntfsclone v1.11.3-WIP" so I should be good to go.

You should also browse the following links:

Linux NTFS Project
Bahut: Cloning XP with Linux and ntfsclone
Knoppix Rescue FAQ
Installing Ubuntu on Fujitsu ST 4000 Tablet PC (they used ntfsclone to backup the WinXP that was on the Tablet PC first)
System Recovery with Knoppix

The hardware that I'm backing up is a Toshiba Tecra 8200 with an 18GB HD that is not going to be very full. At this point, I've finished installing all of the Windows 2000 patches along with some key tools, but it's still not 100% installed. Still, it's a good spot to take a snapshot since everything that is installed is working fine.

Steps:

1. Make sure the system is connected to the network, or that there is a USB drive attached with a FAT32 file system. You'll need a storage location where you can write the image file to. A network folder is preferred so that you can write to an NTFS file system and avoid the 4GB limit of FAT32.

2. Boot into Windows and perform last-minute housekeeping:

a. Clear out any temporary folders/files.
b. Move easily restored user files off of the system.
c. Empty the trash folder.
d. Do a CHKDSK of the file system (requires reboot)
e. Defrag the hard drive.
f. Verify connectivity

Note for step "b.": This image is not meant to be a frequent backup of user files. There should be other tools in place for that (Second Copy 2000, rsync, tar/bzip2). By pulling easily restored user files off of the hard drive, you'll make the image file much smaller and easier to manage. Plus, during a restoration, you'll be overwriting user files in the image with newer user files from the most recent backup anyway.

3. Boot to the Knoppix CD. On the Tecra 8200 you are required to press the [F2] key during the bootup sequence, which will present you with a list of boot sources. You can then press [C] for CD-ROM. On other systems, you may need to muck with the boot order in the BIOS so that the CD-ROM comes before the hard drive.

I will assume that Knoppix boots properly on your system and that you end up at the Knoppix desktop.

4a. If you are going to connect to a network shared folder to store the image file, then look for the penguin icon on the bar at the bottom of the screen labeled "KNOPPIX". Open this menu, go to "Utilities" and then "Samba Network Neighborhood". If Samba (SMB) is working properly, then you should see your local Windows domain and you can browse to your network share.

4b. If you are going to connect to a USB drive for storage of the image, then look for (FIXME).

5. Fire up the command line. To do this in Knoppix, go back to the "KNOPPIX" (penguin) icon and click on the "Root Shell" option. You will see a terminal window open up with a prompt that looks like:

root@1[knoppix]#

6. Familiarize yourself with the partitions on the system:

root@1[knoppix]# cat /proc/partitions
major minor #blocks name
3 0 19535040 hda
3 1 19535008 hda1
8 0 60051600 sda
8 1 32764536 sda1
240 0 1942016 cloop0
root@1[knoppix]#

The above listing shows that there is a ~18GB hard drive installed (hda) with a single partition (hda1) that fills most of the drive. This drive (hda) is what we plan on backing up, if your system uses a different drive letter for the operating system drive, then you will need to adjust later commands. Most ATA/IDE disk drives used for booting Windows are labeled as "hda". You will generally only see "sda" used as the boot drive on systems that boot from SCSI drives.

Notice that there is also a 60GB USB drive attached (sda) with a single 32GB partition (sda1). If I want, I could mount this 32GB FAT32 partition as my destination media to the /tmp/imagedest mount point (see step 7b).

7a. Create a mount point (/tmp/imagedest) and mount the Samba network share that you browsed to earlier. Note the name as shown in the Konqueror browse window (smb://DOMAIN\account@servername/foldername). This is typically case-sensitive, so you'll need to pay close attention to that. Or you could mount the USB drive at this location.

root@1[knoppix]# mkdir /tmp/imagedest
root@1[knoppix]# mount -t smbfs -o username=DOMAIN\\account //servername/foldername /tmp/imagedest
Password: ******
root@1[knoppix]#

Notes:
- You'll need a double-backslash between your domain name and your account name in the Windows domain.
- Domain names in Samba are almost always all upper-case.
- Account names are almost always all lower-case.
- Samba is picky about case.
- You will be prompted for your password.
- The mount command does not give feedback. You can check that there is now a new mount point by repeating the mount command without any arguments.

7b. (FIXME) Show how to mount the USB as the target point.

8. Save the partition table and the master boot record (MBR).

Warning: VERIFY your commands before using them. It's very easy to blow away your operating system by accident when using the following tools.

root@1[knoppix]# sfdisk -d /dev/hda > /tmp/imagedest/myuser-hda.dump
root@1[knoppix]# dd if=/dev/hda bs=512 count=1 of=/tmp/imagedest/myuser-hda.mbr
1+0 records in
1+0 records out
512 bytes transferred in 0.426934 seconds (1199 bytes/sec)
root@1[knoppix]#

Notes:
- I'd recommend replacing "myuser" in the output filenames with a minimum of the date, the model/make of the system being imaged, and possibly the username associated with the system.

9. Use ntfsclone to backup the individual NTFS partitions. Note that this only works for NTFS partitions and may have unpredictable effects if you try to backup a FAT16 or FAT32 partition. You will need to repeat this command for each NTFS partition that you want to save. Notice that we are breaking the image into 4000MB chunks to allow these chunks to be easily placed onto DVD media for archival. You can use smaller chunk sizes if you run into other issues, but it makes it slightly more difficult later to reconstruct the image.

root@1[knoppix]# ntfsclone -s -o - /dev/hda1 | gzip | split -b 1000m - /tmp/imagedest/myuser-diskimage-hda1.img.gz_
root@1[knoppix]#

Notes:
- There are two places in the command where a "-" appears by itself. These are critical as they tell ntfsclone to pipe to standard output ("-o -") and that the split command should pull from standard input ("-" by itself).
- You'll probably want to use the underscore ("_") on the end of the image filename so that split adds the 2-letter suffix (aa, ab, ac, etc) in a way that is not confusing.
- Note, I also hit the 2GB limit (even though I was writing to a share on an NTFS volume). So I went ahead and backed off to 1GB splits.

(FIXME) (to be continued)

NTBackup to a disk file

Note: This post may be incomplete...

Sometimes you have to start simple. For us, after fighting with tape drives and tape drive software, we've switched back to using NTBACKUP that comes with Windows 2000 Server and Windows 2003 Server. We've also given up on tape drives for the moment since our backup needs have drastically outstripped what the old tape drive is capable of.

(Note to self: If you're going to use a tape drive to backup your daily files. Make sure you own (3+) drives of the same make/model that can read tapes from each other. Make sure at least one of those drives is located at an alternate location and is tested weekly. You may want to connect up with another local company who is also using the same technology, such as another company in the same building/town.)

Our backup plan for data files has a few goals and is still a work in progress:

1) Easy restoration when a user says "oops". This means that we need an easy way to restore files that a user has screwed up or accidentally deleted. Possibly even entire folder trees.

We handle this primarily through using "Shadow Copies" in Windows 2003 Server with a sizeable shadow cache setup on another drive within the server. We have roughly 60GB of user data right now, and our cache is 25GB. Shadows are created twice per day (7am and noon) on Monday-Friday and we're getting around 30 days of retention at the moment.

Our secondary plan for dealing with "oops" issues is the weekly full NTBackup job that is written to a central file server. We keep 2 sets of those weekly backups on that central server, giving us 7-14 days of "oops" protection. In addition, since we're backing up to files, we don't have to go scrounging for tapes in order to do a quick file restore. Daily appends also get written to those weekly snapshot folders.

Our third, down-n-dirty method is a workstation that mirrors all changes to a local drive on a nightly basis. This isn't very reliable and loses file attributes and security settings. But it does serve as a last resort in cases where the two primary methods fail.

2) The second goal is off-site storage of backups. This is made easier by putting all of the weekly snapshots and daily updates onto a central server. That allows us to collect the latest versions of the .BKF files and quickly move them to a removeable drive. In addition, the speed of the removable drive matters less because it can be updated after the backup window. The only issue you'll run into is that Windows has difficulty copying large files, so you will have to dig out the GNU Win32 tools (such as the "cp" command) in order to move these multi-gigabyte files around.

So now for the gory details.

Our central backup server is called, appropriately enough, "Backup1" with a single share point called "Backups". Under the backup folder, we have 2 subfolders, one for each week ("Weekly1" and "Weekly2"). This requires creating 2 backup jobs on each server that we want to backup, with the scheduling set to write to the appropriate folder during the proper week. You could also do some scripting to determine which week to write to, but simpler is better for us.

One trick is to store your backup specification file (.BKS) in a central location and use it for all data backups on that server. You'll note that our backup specification file is called "Server2-DataFileBackupSpecification.bks" and gets used by all of the jobs on that server that are backing up user data.

Here's our backup command for Week #1:

C:\WINNT\system32\NTBACKUP.EXE backup "@D:\Data\NTBackup\Server2-DataFileBackupSpecification.bks" /n
"Server2-Weekly1Snapshot" /d "Server2-Weekly1Snapshot" /v:yes /r:no /rs:no /hc:off /m normal /j
"Server2-Weekly1Snapshot" /l:s /f "\\Domain1\Backups\Weekly1\Server2-WeeklySnapshot.bkf"

And the command for Week #2:

C:\WINNT\system32\NTBACKUP.EXE backup "@D:\Data\NTBackup\Server2-DataFileBackupSpecification.bks" /n
"Server2-Weekly2Snapshot" /d "Server2-Weekly2Snapshot" /v:yes /r:no /rs:no /hc:off /m normal /j
"Server2-Weekly2Snapshot" /l:s /f "\\Domain1\Backups\Weekly2\Server2-WeeklySnapshot.bkf"

Notice that the target filename is identical and the only difference is that you store it in the other folder. This allows our offsite system to pull the latest file from either folder without any fancy tricks (other then comparing source/target timestamps).

Microsoft reference links:
Windows 2003 Server - NTBackup Command
Windows XP - NTBackup Command

That takes care of the weekly snapshots (a full backup that also resets the archive bit). For our daily updates, we need to set NTBackup to append to those files and only backup files that have changed since the snapshot was taken.