Network Captures

Version History

Software

MS Network Monitor

[IMG] [IMG]
Comes in two flavors:

Sniffer Pro 4.5

[IMG] [IMG]

Ethereal

[IMG] [IMG]
http://www.ethereal.com/

tcpdump

[IMG]
http://www.tcpdump.org/

dsniff

http://www.monkey.org/~dugsong/dsniff/

ettercap

[IMG] [IMG]
http://ettercap.sourceforge.net/

Captures

1. PPTP Connection fails

Capture (NetMon2)

Description

A Windows client (192.168.1.94) attempts to establish a PPTP connection to a W2K RRAS server (193.121.44.110) via PPTP. The client is asked for an account and password upon connection attempt, but is unable to set up a VPN. The client receives the following message: "Error 721: The remote computer is not responding"

Questions

Explain PPTP (RFC 2637) in terms of protocol(s), port(s), etc.

In order to establish a working PPTP connection, two protocols on top of IP come into play:

Determine the network host to blame for the failing PPTP connection

The step-by-step approach to solving this would be to check if routing between the two hosts works ok, then check the network capture for successful communication over the TCP connection, and finally check for successful communication over the GRE tunnel.

(Note: ARP resolution is not discussed in this example)

Without having to look at every packet in detail, we can see the client initiating a TCP session (SYN), the server responding (SYN+ACK), and the client sending a confirmation (ACK) packet in packets 176, 177, 178:

[IMG]

At this point we can conclude that there is no routing problem between the two hosts, and after looking at the content of the packets that TCP port 1723 is not blocked by any firewall. Since the client has IP address 192.168.x.y, we know that there is a network device that is transparently masquerading/NAT'ting the private address range to an internet-routable address.

Next, we see some TCP/PPTP communication in both directions (packets 178-183, 185). Note that, without looking at the detailed packet info, we can see from the "description" column that communication appears to be normal (no apparent errors):

[IMG]

Packet 186 is interesting, it is the first non-TCP packet sent from client to server. If we look at the packet in detail, we see no sign of any TCP or UDP headers: the IP header is followed by a GRE header, which in turn encapsulates a PPP frame:

[IMG]

(PPP=Point-to-Point tunneling Protocol for point-to-point communication (eg. over phone/serial lines). PPTP=encapsulation of PPP frames in GRE frames that can be transported over an IP network like ethernet.)

Compare this with the headers of a normal TCP packet (packet 183):

[IMG]

where PPTP is the name of the application protocol that runs on top of a TCP session. This is confusing: PPTP=TCP/1723 + GRE, but the TCP/1723 connection is called "PPTP" in NetMon2

Packet 187 is extremely interesting. It is an ICMP message. ICMP messages -apart from "echo request" and "echo reply" packets from the ping program- always indicate that something weird is happening on the network:

[IMG]

The description field clearly states that it considers 193.121.44.110 to be unreachable. It's important to note that the machine that sends this ICMP error message is 192.168.10.254.

Solution: It is clear that 192.168.10.254 is a router that sits somewhere between 192.168.1.94 and 193.121.44.110. It is very likely that this machine performs the NAT'ting of the 192.16.x.y addresses and that it cannot handle the non-TCP/UDP protocol that GRE is, hence the PPTP connection fails.

NAT and PPTP: NAT of TCP and UDP communication is not overly complicated, as the router performing the NAT must simply be able to manage sockets (IP addresses & port numbers) and sequence numbers. The router can identify the connection initiated from the client via the source IP address, source TCP/UDP port, source sequence number, and destination IP address, destination TCP/UDP port, destination sequence number. When the same client initiates another connection to the server, the source TCP/UDP port changes, hence the router is able to distinguish both connections. However, the GRE protocol works differently - it has no notion of ports & sequence numbers.

Imagine two hosts on a LAN behind a masquerading router. If the router were to just translate the IP addresses, it would be impossible for two machines on the LAN to connect to the same PPTP server at the same time: to the PPTP server, both connections would be coming from the same host (the masquerading router) and hence the data stream would confuse the PPTP server. However, there is a mechanism that allows PPTP to be masqueraded: GRE-PPTP has a notion of caller-id and callee-id in the GRE stream to uniquely identify a connection. A masquerading router must have specific tracking code for PPTP-GRE traffic and change the callee-id to successfully masquerade more than one PPTP-GRE connection to the same host. MS Internet Connection Sharing, ISA server and Linux firewalling have this functionality. The CISCO router in this example doesn't.

2. ping btbntsys14 fails (example 1)

Capture (NetMon2)

Description

The command "ping btbntsys14" fails.

Questions

Explain the name resolution process and why the ping fails

The network trace is fairly small. The first "name resolution" packet we see is packet 19. It's a DNS request to 212.8.180.122. Packet 20 shows the reply from the DNS server, saying that it cannot resolve btbntsys14.

[IMG]

Note that:

The next name resolution traffic we see is in packets 27, 30, and 34, where the client is broadcasting NetBIOS Name Service requests over UDP port 137 to all hosts on the local subnet. No hosts reply to these requests.

[IMG]

Solution: The ping command fails because of a name resolution problem. note that it takes over 6 seconds from the first name resolution packet (19) to the last (34).

What is the error message returned by the ping command ?

"Unknown host btbntsys14"

If there was a WINS server on the subnet, what would be its IP address ?

There is no WINS related traffic to be seen in this network capture. Hence, we must look for attempts from the client to reach a machine on the local subnet. Packets 18, 22, 24, 26, 32, 35, and 43 are interesting. They are attempts from the client to reach 192.168.1.250. So, 192.168.1.250 could be a WINS server configured on the client TCP/IP settings.

Packet 21 is an (unanswered) ARP request for 192.168.1.35, but it was not sent from client 192.168.1.94 (different MAC address).

3. ping btbntsys14 fails (example 2)

Capture (NetMon2)

Description

The command "ping btbntsys14" fails.

Questions

Explain the name resolution process and why the ping fails

The packets related to the name resolution of btbntsys14 are:

Solution: Once again, the ping command fails because of a name resolution problem.

What is the error message returned by the ping command ?

"Unknown host btbntsys14"

4. ping btbntsys14 fails (example 3)

Capture (NetMon2)

Description

The command "ping btbntsys14" fails.

Questions

Explain the name resolution process and why the ping fails

The first two packets of interest are 16 and 17. Packet 16 is a name resolution request for the WINS server at 172.16.10.4. In packet 17, interestingly enough, the WINS server successfully responds to the request with IP address 172.16.10.44:

[IMG]

Incidentally, the host immediately sends an ICMP echo request to 172.16.10.44 in packet 18, and subsequent ICMP echo requests in packets 24, 25 and 29:

[IMG]

However, none of the ICMP echo requests gets a reply.

What is the error message returned by the ping command ?

"request timed out"

5. ping btbntsys14 fails (example 4)

Capture (NetMon2)

Description

The command "ping btbntsys14" fails.

Questions

Explain the name resolution process and why the ping fails

The packets related to the name resolution of btbntsys14 are:

Next, in packets 54, 69, 79, and 101, we see the client issuing ARP frames to determine the MAC address of the NIC of the host with IP address 192.168.1.223:

[IMG]

Note that this frame is a sent to a broadcast MAC address. It is not an IP packet, hence there is no source or destination IP address. This also shows when we analyze the content of the frame in greater detail:

[IMG]

In conclusion, even though the name resolution was successful, there is no host that responds to any ARP requests, hence ICMP packets aren't even sent out. In the previous example, ICMP packets were sent out to the MAC address of the default gateway, since the host to ping was on another subnet.

What is the error message returned by the ping command ?

"request timed out"

6. ??? www.linux.org

Capture (NetMon2)

Description

Look for signs of the host "www.linux.org" in the trace.

Questions

What command was typed in at the command prompt ?

The first thing to notice in this network trace is that there are a lot of ICMP messages in the network capture. ICMP messages are always things to look out for in network traces. Apart from ICMP echo request/reply packets, ICMP is used to send error messages that may indicate network (configuration) problems.

In this capture, we see two types of ICMP messages: echo requests (generated by commands such as "ping") and TTL exceeded messages:

[IMG]

Let's look at packet 12 in detail:

[IMG]

The packet was sent with a Time To Live of 1. Every IP packet is sent out with a TTL set to a value between 1 and 255, typically 64 or 128 are used. Every hop that routes the packet, decreases the TTL by 1 and discards the packet when the TTL drops to 0. This prevents routing loops by guaranteeing that packets will be dropped after a finite number of hops, nad hence a finite amount of time. It is an RFC requirement that the hop dropping the packet sends an ICMP error message to the sending host.

A packet sent out with a TTL of 1 will not get past the first hop.

Packets 12, 14, and 16 are essentially the same, as are the error packets 13, 15, and 17. Packets 73, 75, and 77 are ICMP echo requests again, but with a TTL of 2, hence they cause hop 192.168.10.254 to send ICMP TTL exceeded messages in packets 74, 76, and 78. Packets 137 to 142 are the same story, with a TTL=3.

Solution: The command that was run to generate this type of traffic was "tracert www.microsoft.com". tracert.exe sends a series (default 3) of packets to an internet host with rising TTL values. As the intermediate hops discard packets, they send ICMP error messages back. tracert.exe interprets these messages and is able to determine the routing path followed by IP traffic between the two hosts.

From what type of machine (Operating System) was this command run ?

It was run from a Windows operating system. The windows tracert.exe executable uses ICMP echo request packets to trigger the TTL exceeded messages.

What is the end result of the command ?

The end result looks something like this:

C:\>tracert www.linux.org

Tracing route to www.linux.org [198.182.196.56]
over a maximum of 30 hops:

  1   <10 ms   <10 ms   <10 ms  192.168.1.24
  2    10 ms    10 ms   <10 ms  192.168.10.254
  3    10 ms    10 ms    10 ms  212.8.180.91
  4    20 ms    10 ms    20 ms  212.8.191.124
  5    10 ms    10 ms    10 ms  212-8-180-161.bru01.btconnect.be [212.8.180.161]
  6    10 ms    50 ms   <10 ms  serial6-1.bru-icr-01.carrier1.net [212.4.203.13]
  7    10 ms    10 ms    10 ms  fastethernet6-0.bru-bbr-02.carrier1.net [212.4.199.209]
  8    10 ms    10 ms    10 ms  pos1-0.bru-bbr-01.carrier1.net [212.4.200.73]
  9    20 ms    21 ms    20 ms  pos13-1.lon-bbr-02.carrier1.net [212.4.199.61]
 10   100 ms   100 ms   100 ms  pos13-0.nyc-bbr-02.carrier1.net [213.239.20.254]
 11   120 ms   120 ms   110 ms  gigabitethernet1-0.nyc-pni-02.carrier1.net [212.4.193.198]
 12   120 ms   110 ms   120 ms  500.POS2-1.GW14.NYC4.ALTER.NET [157.130.94.249]
 13    91 ms   100 ms   100 ms  578.ATM4-0.XR1.NYC4.ALTER.NET [152.63.26.246]
 14   121 ms   100 ms   120 ms  189.at-2-0-0.TR1.NYC8.ALTER.NET [152.63.21.98]
 15   141 ms   120 ms   130 ms  124.ATM7-0.TR1.TOR2.ALTER.NET [152.63.7.74]
 16   131 ms   140 ms   130 ms  199.ATM7-0.XR1.TOR2.ALTER.NET [152.63.128.37]
 17   120 ms   120 ms   160 ms  POS6-0.GW4.TOR2.ALTER.NET [152.63.131.137]
 18   130 ms   130 ms   130 ms  att6-gw.customer.alter.net [208.217.112.78]
 19   141 ms   120 ms   130 ms  srp2-0.core1-tor.bb.attcanada.ca [216.191.65.241]
 20   120 ms   120 ms   120 ms  pos8-0.core1-ott.bb.attcanada.ca [216.191.65.178]
 21   161 ms   140 ms   140 ms  pos5-0-0.hcap1-ott.bb.attcanada.ca [216.191.225.2]
 22   150 ms   130 ms   140 ms  invlogic1.p2p.attcanada.ca [216.191.132.146]
 23   140 ms   140 ms   140 ms  router.invlogic.com [207.245.34.122]
 24   150 ms   140 ms   130 ms  www.linux.org [198.182.196.56]

Trace complete.
      

How long does it take to execute the command ? Why ? Improve performance.

It takes over 85 seconds to complete the tracert.exe command. tracert.exe, when run without any additional parameters, will perform a reverse DNS lookup on every intermediate hop it finds, in order to find out its name. These lookups can take a while, especially on IP addresses that have no reverse DNS mapping, like the first 4 hops in the tracert.

If you just want a trace of the route to a host, use: tracert -d www.linux.org.

7. ??? www.microsoft.com

Capture (tcpdump)

Description

Look for signs of the host "www.microsoft.com" in the trace.

Questions

What command was typed in at the command prompt ?

Once again, we see a lot of ICMP TTL exceeded messages in response to consecutive messages sent with rising TTL values:

19:02:45.604546 192.168.1.222.48671 > 207.46.230.219.33435:  [udp sum ok] udp 10 [ttl 1] (id 48672, len 38)
19:02:45.605472 192.168.1.24 > 192.168.1.222: icmp: time exceeded in-transit for 192.168.1.222.48671 > 207.46.230.219.33435:  udp 10 [ttl 1] (id 48672, len 38) [tos 0xc0]  (ttl 255, id 26864, len 56)
...
19:02:45.608393 192.168.1.222.48671 > 207.46.230.219.33438:  [udp sum ok] udp 10 (ttl 2, id 48675, len 38)
19:02:45.611259 192.168.10.254 > 192.168.1.222: icmp: time exceeded in-transit for 192.168.1.222.48671 > 207.46.230.219.33438:  udp 10 [ttl 1] (id 48675, len 38) [tos 0xc0]  (ttl 254, id 32232, len 56)
...
19:02:45.618040 192.168.1.222.48671 > 207.46.230.219.33441:  [udp sum ok] udp 10 (ttl 3, id 48678, len 38)
19:02:45.622765 212.8.180.91 > 192.168.1.222: icmp: time exceeded in-transit for 192.168.1.222.48671 > 207.46.230.219.33441:  [bad udp cksum e6c5!] udp 10 [ttl 1] (id 48678, len 38) [tos 0xe0]  (ttl 253, id 64300, len 66)
...
19:02:45.632864 192.168.1.222.48671 > 207.46.230.219.33444:  [udp sum ok] udp 10 (ttl 4, id 48681, len 38)
19:02:45.638367 212.8.191.124 > 192.168.1.222: icmp: time exceeded in-transit for 192.168.1.222.48671 > 207.46.230.219.33444:  udp 10 [ttl 1] (id 48681, len 38) [tos 0xc0]  (ttl 252, id 28412, len 56)
...
19:02:45.650058 192.168.1.222.48671 > 207.46.230.219.33447:  [udp sum ok] udp 10 (ttl 5, id 48684, len 38)
19:02:45.655882 212.8.180.134 > 192.168.1.222: icmp: time exceeded in-transit for 192.168.1.222.48671 > 207.46.230.219.33447:  udp 10 [ttl 1] (id 48684, len 38) (ttl 251, id 55369, len 56)
...
19:02:45.667081 192.168.1.222.48671 > 207.46.230.219.33450:  [udp sum ok] udp 10 (ttl 6, id 48687, len 38)
19:02:45.673103 212.4.203.13 > 192.168.1.222: icmp: time exceeded in-transit for 192.168.1.222.48671 > 207.46.230.219.33450:  udp 10 [ttl 1] (id 48687, len 38) (ttl 250, id 58456, len 56)
...
      

It's important to note that the messages that are triggering the ICMP TTL exceeded errors are no ICMP echo requests this time: they are instead UDP packets sent to high port numbers (>32K) on the destination system:

19:02:45.604546 192.168.1.222.48671 > 207.46.230.219.33435:  [udp sum ok] udp 10 [ttl 1] (id 48672, len 38)
19:02:45.605472 192.168.1.24 > 192.168.1.222: icmp: time exceeded in-transit for 192.168.1.222.48671 > 207.46.230.219.33435:  udp 10 [ttl 1] (id 48672, len 38) [tos 0xc0]  (ttl 255, id 26864, len 56)
...
      

From what type of machine (Operating System) was this command run ?

This command was typed from a UNIX/Linux operating system: traceroute -n www.microsoft.com

What is the end result of the command ?

At one point, ICMP TTL exceeded packets are returning. This behavior can be explained by a border router/firewall from Microsoft filtering out protocols like ICMP, UDP to high ports.

The command result looks like this:

[root@sneppef /root]# traceroute -n www.microsoft.com
traceroute: Warning: www.microsoft.com has multiple addresses; using 207.46.230.219
traceroute to www.microsoft.akadns.net (207.46.230.219), 30 hops max, 38 byte packets
 1  192.168.1.24  0.976 ms  0.942 ms  1.035 ms
 2  192.168.10.254  2.863 ms  2.916 ms  2.941 ms
 3  212.8.180.91  4.904 ms  5.017 ms  4.536 ms
 4  212.8.191.124  5.743 ms  5.314 ms  5.269 ms
 5  212.8.180.134  5.327 ms  5.435 ms  5.578 ms
 6  212.4.203.13  5.939 ms  6.171 ms  6.699 ms
 7  212.4.199.209  7.387 ms  7.688 ms  5.701 ms
 8  212.4.200.73  6.615 ms  8.197 ms  7.595 ms
 9  212.4.199.61  14.092 ms  14.677 ms  14.167 ms
10  212.4.211.130  117.105 ms  115.443 ms  116.296 ms
11  212.4.193.202  115.697 ms  115.101 ms  114.638 ms
12  4.24.163.17  115.167 ms  115.204 ms  116.500 ms
13  4.24.7.1  112.508 ms  112.031 ms  113.606 ms
14  4.24.10.210  113.516 ms  112.842 ms  113.476 ms
15  4.24.10.177  114.252 ms  114.336 ms  114.843 ms
16  4.24.10.182  128.915 ms  129.399 ms  129.003 ms
17  4.24.10.154  132.101 ms  132.600 ms  132.266 ms
18  4.24.10.213  334.922 ms  217.811 ms  169.394 ms
19  4.24.10.114  135.744 ms  135.481 ms  134.766 ms
20  4.24.5.61  172.486 ms  180.532 ms  174.065 ms
21  4.24.8.25  197.385 ms  194.629 ms  194.826 ms
22  4.24.4.177  214.100 ms  212.688 ms  213.598 ms
23  4.24.4.78  214.794 ms  214.519 ms  213.902 ms
24  4.24.5.102  198.322 ms  198.836 ms  197.886 ms
25  4.24.125.66  193.762 ms  192.731 ms  192.926 ms
26  207.46.129.175  190.405 ms  190.748 ms  191.682 ms
27  207.46.190.26  178.418 ms  179.661 ms  178.588 ms
28  * * *
29  * * *
30  * * *
      

8. What's happening on this internet host (example 1)

Capture (tcpdump)

Description

...

Questions

What is happening here ?

9. What's happening on this internet host (example 2)

Capture (tcpdump)

Description

...

Questions

What is happening here ?

...

What is the subtle difference with the previous example ?

...

10. webmail login slow

Capture (tcpdump)

Description

A web server with two IP addresses (212.8.180.38 (frontend) and 172.16.10.38 (backend)) hosts a hotmail-like application over HTTP. The user mail itself is served via IMAP servers. The web application is slow.

Questions

Find a 3-way TCP handshake in the tcpdump output

...

How many IMAP servers are there ?

...

Troubleshoot the performance problem of the webserver and suggest improvement(s)

...

Sniff a couple of IMAP accounts and passwords from the wire

...

(Difficult) For every logon, the web server queries a backend server over a specific protocol: Give server and protocol

...

11. what's happening on this LAN

Capture (SnifferPro)

Description

This is a network capture from a LAN that is experiencing network performance problems.

Questions

Find the problem on the LAN and suggest a solution

...

12. *tough* HTTP problem from script

Capture (NetMon2)

Description

This is a trace from a client computer that is attempting to access a webserver via a script, over an HTTP proxy.

Questions

What type of error is received by the client and who is generating the error ?

...

What do the RFCs say about this behavior ?

...

13. ping on subnet fails

Capture (NetMon2)

Description

A ping from 192.168.0.10 to 192.168.10.10 of an address on the LAN fails, even though 192.168.10.10 is up and connected to the network. A network capture was taken from host 192.168.0.1 with MAC address 002018B81DD0.

Questions

Explain why the ping fails and offer a solution to this problem

...

14. ping on subnet fails sometimes

Capture (NetMon2)

Description

See previous example

Questions

Explain the different network behavior you can observe from the network capture

...

Find the registry key that controls this behavior

...

What is the difference between IP routing and IP forwarding ?

...

Find the registry key that controls IP forwarding

...

15. ping to 207.21.36.81 fails

Capture (NetMon2)

Description

This is a network capture from a LAN.

Questions

What is happening on the network ?

...

What is the error message on the client side ?

...

16. surfing is slow

Capture (NetMon2)

Description

(Note that due to a minor bug with W2K+SMS Netmon2, locally generated packets are repeated twice) When surfing, the initial connection to a site is always slow.

Questions

Explain why the initial connection is slow

...

17. ftp to port 1000 fails

Frontend Capture (NetMon2)
Backend Capture (NetMon2)

Description

FTP to host 193.121.44.112 port 1000 from a host behind a Windows 2000 ICS doesn't work; the client can log in successfully, but is unable to upload/download any files or list a directory's content.

Questions

Without looking at the capture, explain why the file transfers are failing

...

What's the exact reason for the failure in this sniff ?

...

18. cannot surf to www.zdnet.com

Capture (NetMon2)

Description

Host 192.168.1.222 cannot surf to www.zdnet.com. Other sites like www.google.com work just fine. Also, all the other machines in the office are able to surf to www.zdnet.com, so the site is definately not down.

Questions

By looking at the network capture, what is a possible cause of this problem ?

Packet 40 sees the host 192.168.1.223 sending out a DNS query for the host www.zdnet.com; packet 41 contains the answer from the DNS server. Packet 42 shows host 192.168.1.223 sending out a TCP SYN packet to 205.181.112.65 (www.zdnet.com). However, there is never a reply from 205.181.112.65: no SYN/ACK packet to continue the three-way TCP handshake, not a RST packet or ICMP error message. The fact that we receive no response whatsoever indicates that there is some kind of filtering/firewalling going on.

[IMG]

Yet, host 192.168.1.223 is able to surf to other sites, and other clients from the 192.168.1.0/24 network are able to surf to www.zdnet.com. One possibility is that something inside packet 42 is leading a firewall to believe that 192.168.1.223 is not sending a valid TCP packet.

If we look more closely at the packet's TCP header, we notice the following:

[IMG]

Some bits in the TCP header are non-zero, when SMS Network Monitor denotes them as "Must-Be-Zero" (MBZ). Before drawing any hasty conclusions, we should look at the same packet with a network monitor with more advanced protocol analysis, Ethereal:

[IMG]

Notice how Ethereal marks the two bits that SMS Netmon tags as MBZ as "CWR" and "ECN" bits. This is the solution to our problem; Host 192.168.1.223 is using Explicit Congestion Notification, a

Links