More about securing our communication with the Internet.
Part 1 of this article described the theory. Now let's pick up the VPN mini-HOWTO, study the script that creates the VPN, run it, and explain what we see. The HOWTO (ftp://metalab.unc.edu/pub/Linux/docs/HOWTO/mini/VPN, text version, or metalab.unc.edu/mdw/HOWTO/mini/VPN.html) is required reading for this article. The script, by Miquel van Smoorenburg and Ian Murdock, is in section 4.10 of the HOWTO.
The script runs on the local VPN server. If you configure reciprocally, you can arrange to let it run from either side with functionally equivalent results. What I call local/remote, Arpad Magosanyi calls master/slave. For working purposes, ssh needs a remote user account under which to log in. So, a remote user “slave”, which doesn't correspond to any human user, is first created and configured (permissions, keys).
The heart of the script embodies running pppd and route as described in Part 1 (section entitled The Network). With simplifications, here it is. Line 33 is the centerpiece.
VPN HOWTO script:
line 1: #! /bin/sh line 19: MYPPPIP=192.168.0.1 line 20: TARGETIP=192.168.0.2 line 21: TARGETNET=193.6.37.0
PPPD on both machines:
line 33: /bin/pty-redir /usr/bin/ssh -o\
'Batchmode yes' -t -l slave 206.170.217.204\
/usr/local/bin/sudo /usr/sbin/pppd >/tmp/device
line 34: TTYNAME='cat /tmp/device'
line 39: /usr/sbin/pppd $TTYNAME\
${MYPPPIP}:${TARGETIP}
ROUTE on both machines:
line 45: route add -net $TARGETNET gw $TARGETIP line 46: /usr/bin/ssh -o 'Batchmode\ yes' -l slave 206.170.217.204\ /usr/local/bin/sudo /home/slave/sshrouteI numbered the lines, then deleted extraneous ones. For readability, I also substituted shell variables and changed their values to reflect real file and path names on my Red Hat 5.1 servers. Ignore sudo in line 33—it is nothing more than a command filter/authenticator, a mechanism you set up to allow some commands to run and disallow others. It is here for additional security. It is set up to permit pppd (HOWTO section 4.9), so its absence from line 33 would not affect operations. When you read line 33, you should see
ssh -t -l slave 206.170.217.204 pppdThis says, “get logged into the remote machine under its user account 'slave' and run pppd; make the remote machine set up a pseudo-terminal (-t) as the destination for output of this pppd process.”
Please note that in section 6 of the HOWTO, “Doing it by hand”, unlike the script, -t is absent from the command line. When I first did it by hand, I was unsuccessful at getting the desired “garbage right into [my] face” until I added -t. I got it going by using the following machine Internet addresses:
local machine's public IP: 206.170.218.30 remote machine's public IP: 206.170.217.204
The screen capture from the local machine was:
[root@localhost /root]# ssh -t -l slave 206.170.217.204 /usr/sbin/pppd ~˜}#A!}!}!} }2}!}$}"(}%}&} } öŒ}'}"}(}""q~~˜}#A!} !}!} }2}!}$}"(}%}&} } öŒ}'}"}(}""q~~˜}#A!}!}!} }2}!}$}"(}%}&} } öŒ}'}"}(}""q~~˜}#A!}!}!} }2}!}$} "(}%}&} } öŒ}'}"}(}""q~~˜}#A!}!}!} }2}!}$}" (}%}&} } öŒ}'}"}(}""q~~˜}#A!}!}!} }2}!}$}"(}%}&} } öŒ}'}"}(}""q~~˜}#A!}!}!} }2}!}$}"(}%}&} } öŒ}'} "}(}""q~~˜}#A!}!}!} }2}!}$}"(}%}&} } öŒ}'}"} (}""q~~˜}#A!}!}!} }2}!}$}"(}%}&} } öŒ}'}"} (}""q~~˜}#A!}!}!} }2}!}$}"(}%}&} } öŒ}'}"} (}""q~~. Connection to 206.170.217.204 closed. [root@localhost /root]#and the simultaneous remote-machine log entries were:
Nov 7 20:17:19 localhost sshd[1403]: log: Connection from 206.170.218.30 port 1023 Nov 7 20:17:21 localhost sshd[1403]: log: RSA authentication for slave accepted. Nov 7 20:17:22 localhost sshd[1405]: log: executing remote command as user slave Nov 7 20:17:22 localhost pppd[1405]: pppd 2.3.3 started by slave, uid 507 Nov 7 20:17:22 localhost kernel: registered device ppp1 Nov 7 20:17:22 localhost pppd[1405]: Using interface ppp1 Nov 7 20:17:22 localhost pppd[1405]: Connect: ppp1 <--> /dev/ttyp0 Nov 7 20:17:52 localhost pppd[1405]: LCP: timeout sending Config-RequestsAugmenting the command with the author's pty-redir in the next step produced:
[root@localhost /root]# /bin/pty-redir /usr/bin/ssh -t -l slave 206.170.217.204 /usr/sbin/pppd /dev/ttyp0[root@localhost /root]# root@localhost /root]#(where did the garbage go?) and simultaneous remote-machine log entries:
Nov 7 20:21:43 localhost sshd[1406]: log: Connection from 206.170.218.30 port 1023 Nov 7 20:21:46 localhost sshd[1406]: log: RSA authentication for slave accepted. Nov 7 20:21:46 localhost sshd[1408]: log: executing remote command as user slave Nov 7 20:21:46 localhost pppd[1408]: pppd 2.3.3 started by slave, uid 507 Nov 7 20:21:46 localhost pppd[1408]: Using interface ppp1 Nov 7 20:21:46 localhost pppd[1408]: Connect: ppp1 <--> /dev/ttyp0 Nov 7 20:22:16 localhost pppd[1408]: LCP: timeout sending Config-RequestsSo far, it's working. Tracing through the log, sshd hears ssh. sshd agrees to run the command requested by ssh (since I preconfigured keys across the machines). The requested pppd command is then run, which prepares to use an interface called ppp1 and associates it with pseudo-terminal /dev/ttyp0. The process stopped there only because pppd was never run from the other end; however, everything here on the remote side went right. We'll see the process consummated below when we run the full-blown script to completion.
What's the difference between these two invocations? On the remote side, nothing; the logs are the same. On the local side, in the second invocation, the entire prior command string was fed to pty-redir and executed under its control, resulting in the garbage going away, it seems. Actually, it just went elsewhere; pty-redir “redirects” it. pty-redir is a short C language program by Mr. Magosanyi. It identifies a pseudo-terminal device that is not in use and opens it. Then it forces standard output—normally directed to the console—to the pseudo-terminal instead. Anything the program (ssh) would send to the console gets diverted to the pseudo-terminal device.
Don't confuse this pseudo-terminal with the one created by the -t option—they're on different machines. ssh -t makes sshd on the remote side create a pseudo-terminal over there, whereas pty-redir operates on the local side. By number, both pseudo-terminals happen to be /dev/ttyp0 this time, but that won't always be so. The local pseudo-terminal is manifested above in the screen output, the remote one in the log.
While it's nice to see the “garbage in our face” for diagnostic purposes, it's better to keep it out of sight for production purposes. That's why pty-redir was written. The pseudo-terminal will prove convenient as the “receiving vessel” for the incoming pppd output stream. We can launch a local pppd into it in order to create the desired connection, instead of having to do it more intrusively at our real terminal.
This is the place to contrast connections made by pppd versus ssh and other daemons. The VPN uses both symmetrical and asymmetrical connections to achieve “tunneling”. ssh is asymmetrical and uses the TCP/IP service-port connections; pppd is symmetrical and uses device-port connections.
The PPP HOWTO section 14, “Setting up the PPP connection manually”, illustrates pppd's requirements. It has to run simultaneously on both computers: the two output streams “meeting in the middle”, so they can negotiate the connection setup. pppd's output stream, visible above, may look like garbage; however, it is pppd's signature and is very meaningful to another copy of pppd. (The PPP HOWTO section 9.5 introduces “ppp garbage”.)
“Meeting in the middle” means getting opposite pppd's to run over terminal or port devices (/dev/ttyX) that connect, with the output of each coming in as input to the other. The idea is to get the incoming data stream fed into an identifiable local device, then launch a local pppd into that same device. The outgoing stream moves into the same “pipeline” from which the incoming one emerges. Any available terminal or serial port device will do. Since Linux provides /dev/ttypx pseudo-terminals that are serial port emulators, they will work just as well. That's essentially what the pty-redir program is doing; it arranges for opposing pppd data streams to meet in the middle. Once that happens, handshaking and negotiation ensue, ultimately manifesting as a pppx interface viewable with the ifconfig command.
This highlights one big difference between pppd and other daemons: symmetry. Last month, we said daemons belong to a matched pair of distinct programs: one the client, one the server or daemon. With pppd there's no distinction, i.e., no client. The program pppd talks to is always another copy of pppd itself, a conversation symmetrical with itself rather than asymmetrical with a different program.
The other difference is that other daemons such as sshd talk over TCP/IP connections involving TCP/IP ports. pppd talks over a connection and uses ports too, but it builds its own kind of connection with its own protocol, not TCP/IP. It uses port devices, not the numbered service ports (sockets) of TCP/IP.
Since computers can multitask, TCP/IP provides for simultaneous conversations between multiple pairs of processes on any given pair of computers. The conversations are broken into data packets. Each packet has a label (header) bearing the destination computer's IP address and port number. Of the (possibly) many conversations the destination computer might be conducting, the port number identifies the one to which this packet belongs. A conversation is uniquely defined by a set of four items in the header: IP address at each end and TCP port number at each end. The packets look like those in Figure 1 (simplified). See “Introduction to the Internet Protocols” at http://oac3.hsc.uth.tmc.edu/staff/snewton/tcp-tutorial/.
Figure 1
In the conversation logged above, we invoked ssh on the command line and directed it to remote computer 206.170.217.204. The first line in the remote log shows:
Nov 7 20:21:43 localhost sshd[1406]: log: Connection from 206.170.218.30 port 1023
ssh launched a packet to sshd on the remote computer. sshd is set up to “reside” at port 22, as ssh knows. The packet it launched looked like that in Figure 2.
Figure 2
sshd on remote port 22 heard this and answered with packets of its own, swapping the above destination and port values left to right. A conversation was underway. The main order of business was encryption key negotiation and authentication, but before getting into that, it looks like sshd told ssh to change port numbers from 22 to 1406 (according to that log entry, which is “signed” by sshd). Thereafter, a flurry of packets like those shown in Figure 3 flew back and forth.
Figure 3
Certain server applications conventionally run on defined port numbers, known publicly so that clients have a place to contact them. These are called “well-known” or “privileged” ports and range from 0 to 1023. (See /etc/services. Port 80 is reserved for httpd, for instance, port 21 for ftpd.) Once contacted, servers often ask clients to switch port numbers to clear its well-known port for other incoming conversations. Port 22 is the main sshd switchboard. The operator there offloads you to another line as quickly as possible (“Please hold while I transfer you”), so she can handle other incoming calls.
It should now be clear that pppd could never use TCP/IP service ports, since no TCP/IP connection generally exists when pppd runs. The connection pppd has to work with is physical—a phone line or direct serial cable. TCP/IP is a transport-layer protocol. pppd (based on HDLC) runs at the lower data-link communications layer, and pppd packets are carriers or envelopes for TCP/IP (or other) packets. Because of these fundamental differences, the overlapping references to “daemons” and “ports” is unfortunate.
Now let's run the script to automate and complete what we did by hand. Here is a local screen capture:
[root@localhost /root]# /etc/rc.d/init.d/VPN start setting up vpn tty is /dev/ttyp1 [root@localhost /root]#
The simultaneous remote log is shown in Listing 1, which shows that sshd agrees to run the command requested by ssh; sshd puts the conversation on port 1022 this time. The requested command (see VPN HOWTO script line 33) is sudo, not pppd. Within the remote machine, sudo in turn runs pppd, which prepares to use an interface called ppp2 and associates it with pseudo-terminal /dev/ttyp1. Just about this time, somebody runs pppd at the local machine (see line 39). Two pppd's meet in the middle, and the local pppd (through ssh, then sshd) tells the remote pppd what IP address numbers it wants to use for each end of this connection. Remote pppd agrees, and records those addresses in the log. At that point, the secure link is in place, bilaterally. The script then proceeds to run route. It informs each VPN server by address which additional machines are available to be reached through the new connection beyond the other server (i.e., the address of the network the opposite server is on). In reaction to script line 46, the log shows sshd reinvoked under new process ID number 1439, this time to have sudo run sshroute, a script (see end of HOWTO section 4.10) that contains the appropriate route command.
The dust has settled. The secure link is now in place. How can we see and use it? It takes the form of a new PPP interface. See it by running ifconfig. Use it by referring certain addresses to it in the routing table (already done), then referring applications to those addresses.
ifconfig now shows two ppp interfaces on each machine, old and new. A screen capture on the local machine looks like that in Listing 2. A screen capture on the remote machine is shown in Listing 3. Local-machine's ppp1 and remote-machine's ppp2 are opposite ends of the same connection, or interfaces to one another. This connection is the one just constructed. Note interface number assignment is machine-specific, and the numbers need not be identical. Local and remote ppp0, the machines' respective ISP connections, aren't directly related.
To refer to the other, each VPN server now has a choice of IP addresses. Local-machine can still contact the remote machine at its public Internet address 206.170.217.204, but has the additional option of calling it 192.168.0.2. If the remote machine is running a web server, for example, a browser on the local machine will pull up exactly the same page by addressing itself to http://206.170.217.204 or http://192.168.0.2. The remote machine can be pinged using either address. To all appearances, our two ppp interfaces seem independent and equivalent. Logically, they are, and you can use them as if they were; physically, they are not.
What's the difference? Primarily, that packets to 192.168.0.2 don't travel on the same footing as those to 206.170.217.204. Rather, the former are carried as data “freight” inside the latter. That's why I depicted the PPP connection as tributary to the public ISP connection in the network diagram in Part 1. The arrangement is called tunneling, because once freight packets arrive at their destination, they emerge from their enclosing packets as from a tunnel. They're released onto the destination network as functional packets, not just passive data.
You may have noticed that the 192.168.0.0 IP addresses chosen for the secure link belong to the “reserved” range prohibited for use on the Internet (PPP HOWTO section 2), yet we are using these on the Internet. Tunneling is what allows us to get away with it. While on the Internet, packets bearing these addresses travel only as data. The Internet need not route them according to those addresses—they piggyback on packets addressed legitimately.
Tunneling enters the discussion of all VPN protocols. Some reflect it in their name, such as Microsoft's PPTP or Point-to-Point Tunneling Protocol. Tunneling is a characteristic of VPNs; if the data being tunneled also gets encrypted, you have a VPN.
That's the other difference here—encryption. Ping packets and web pages going to and from 192.168.0.2 get encrypted and decrypted. Those going to 206.170.217.204, even though it is the same place, do not. But you would not know about the encryption or tunneling, because they're transparent. Since both are present, this is a VPN.
I found the HOWTO's treatment of routing less than comprehensive, perhaps beyond its intended scope. It goes as far as informing each VPN server about the population of workstations on the opposite network, but it doesn't teach those workstations the reverse, and it doesn't provide for end-to-end awareness by workstations on one network about the workstations on the other.
The provided routing from the main script
route add -net 193.6.37.0 gw 192.168.0.2
puts an entry in the local VPN server's routing table that says “If you handle any packets whose address starts with 193.6.37, the clearinghouse for them is 192.168.0.2 (the remote VPN server), send them to him. He'll know what to do with them”, since that's the address of the network to which he belongs.
When I tried to ping a 193.6.37 from the local VPN server, it failed. While I had a route to those machines, the problem was their ignorance of a reverse route back to me. They got my ping packets all right, but they couldn't answer back with a reply packet because their routing table had never heard of me—dead-letter office. So, I never saw the evidence of my half-success.
There are two solutions. One, an entry in each of their routing tables that points the way, unfortunately requires separate adjustments on these multiple machines. The other requires a single change at the remote VPN server, using the proxyarp option of pppd.
ARP (address resolution protocol) is for use in an Ethernet environment. It's insufficient for workstations that talk over Ethernet to know one another's IP address. The only way they can trade TCP/IP packets is inside of Ethernet packets. Those go nowhere unless the destination Ethernet (not IP) address is known.
Machines maintain a directory for looking up a machine's Ethernet address, given its IP address. (The Linux arp command prints it.) If a desired target IP isn't in your list, you automatically emit an ARP broadcast to the network. If any other machine can come up with the matching Ethernet address, it will send it to you.
The pppd proxyarp command makes a white-lie entry in the remote VPN server's directory that equates the local VPN server's TCP/IP address with the remote VPN server's Ethernet address. This fixed my ping problem. If the remote workstations can get replies back to the remote VPN server, they can certainly get them the rest of the way back to me at the local VPN server. (See section 9 of “Introduction to the Internet Protocols” for more about ARP.)
The other routing problem is that local and remote workstations don't have routes to each other. The fix is an entry for each local workstation specifying that either the local or remote VPN server offers the route to the remote network. For each remote workstation, either the remote or the local server offers the route.
For local workstations, the command is
route add -net 193.6.37.0 gw 192.168.0.1 or route add -net 193.6.37.0 gw 192.168.0.2
For remote workstations,
route add -net 193.6.35.0 gw 192.168.0.2 or route add -net 193.6.35.0 gw 192.168.0.1For local workstations,
route add -net 193.6.37.0 gw 192.168.0.1 route add -net 193.6.35.0 gw 192.168.0.2 or route add -net 193.6.37.0 gw 192.168.0.2 route add -net 193.6.35.0 gw 192.168.0.1(The second one works only if you invoked proxyarp when you ran pppd.) For Microsoft machines on either net, give that net's VPN server as the Gateway under TCP/IP properties.
Now you are a VPN expert, so you also know the VPN is worthless unless the computer where it runs is secured in other ways. For instance, rsh and rlogin had better be disabled. ssh was designed to replace them. If they're still hanging around, you have one door double-bolted and another wide-open. Other services like telnet and ftp should be turned off, too. You can do that in the inetd.conf file. And, firewall rules should be deliberately applied. Though beyond the scope of this article, these important considerations require mention.