Network Testing Methodology
To test the ASUS RT-AC66U Router I used two similar test applications, both of which are specifically designed for testing network throughput. The first one was Passmark Performance v7.0 Advanced Network Test. The second one was TamoSoft Throughput Test v1.0, Build 19. These tests measure throughput between two PCs connected through a router (or switch), and normally, the router or switch is the device that is under test. With a known baseline for the router or switch, you can also test the performance of other devices in the communication chain, such as wireless adapters or network interface cards (NICs). In order for this test to work one PC must be set up as ‘Client’ and the other must be set up as the ‘Server’. Each test was run at least five times with the highest and lowest result omitted and the remaining results averaged to give a final result.
During earlier testing of GbE and 10GbE network switches, I’ve already established a baseline benchmark speed for these two workstations, Test System #1 and #2, below. Both of them are Quad-Core Intel systems, and can easily sustain Gigabit transfer speeds and higher, in wired mode. In the past, I used all default values for this benchmark, as shown in the screenshot below, but I discovered in this round of testing that certain combinations required a longer settling time before the true steady-state throughput was revealed. The screens for the Passmark Performance test are the same at both ends, with different elements grayed out, depending on whether you are sitting at the Client computer or on the Server side. The TamoSoft application has different screens for the Client and Server sides.
This benchmark eliminates most of the variables involved in network speed testing, but not all. The PCs themselves can introduce spurious issues, such as hardware bandwidth limitations, resource conflicts, wait states, and buffer inconsistencies. In some cases, the networking hardware is having issues communicating with other networking gear. The following chart shows why you have to dig a little deeper than just looking at the Average Transmission rate that is displayed on the main screen. For some reason, at the start of this test the two systems were having a hard time establishing a rapport. The average value (shown in yellow) doesn’t really reflect the true capability of the network until things start to settle down around the two minute mark. The default time period that’s set in the software for this benchmark is 20 seconds (a looooong time in network transmissions), and the calculated average result at that point is more than 50% higher than the real number. I maxed out the test period to 200 seconds, and was finally able to see some convergence on the data rate after the first minute or so. BTW, the chart below is from a UDP test, where an awful lot of data was NOT making it across; more than 300Mbps was being sent out, and less than 20Mbps was being received and accepted. NOT a good result, as you’ll see….
For the second set of tests, I used an application that I discovered when trying to get the measure of some 10GbE networking gear that was on the test bed. Many of the current test products had difficulty measuring the full performance capabilities of a 10GbE network, but I did find a couple of applications that showed promise for normal test speeds. TamoSoft Throughput Test v1.0, Build 19 was the best of the bunch, especially in its ease of use and real-time graphical display of throughput. The graphs I show from Passmark Performance v7.0 Advanced Network Test are only available for viewing after the test is complete. Plus, there is a 200 second limit for the Passmark software, and I was able to run the TamoSoft test for as long as I wanted. Oh, and did I mention that you can run the TCP/IP and UDP tests at the same time? You can specify that only TCP data be used for the test, and I did run several tests to confirm that there were no differences, when running the UDP tests alongside the TCP one. Results for both upload and download are presented, and I ended up using the two download averages for reporting purposes. In the run below, that means a TCP result of 115.18 Mbps and a UDP result of 421.11 Mbps would be counted as the results. The packet loss (19.1 %) and Round-trip time (2.0 ms) are shown as well, but I didn’t report that. If the readers really want that data, let me know in the comments section and I’ll see about including it, somehow. TamoSoft has a bunch of additional test applications that are more comprehensive and are available for purchase, but this one is free for downloading.
Test System 1 (Server)
- Motherboard: MSI Z68-Express Z68A-GD80 (1.23.1108 BIOS)
- System Memory: 4x 4GB Corsair Vengeance LP DDR3 1600MHz (9-9-9-24)
- Processor: Intel Core i5-2500K Sandy Bridge 3.3GHz (BX80623I52500K)
- CPU Cooler: Thermalright Venomous-X (Delta AFB1212SHE PWM Fan)
- Video: Intel HD Graphics 3000
- Drive 1: OCZ Agility3 SSD 120GB (AGT3-25SAT3-120G)
- Enclosure: Lian Li Armorsuit PC-P50R
- PSU: Corsair CMPSU-750TX ATX12V V2.2 750Watt
- Monitor: SOYO 24″; Widescreen LCD Monitor (DYLM24E6) 1920X1200
- Operating System: Windows 7 Ultimate Version 6.1 (Build 7600)Results
Test System 2 (Client)
- Motherboard: ASUS P7P55D-E Pro (1002 BIOS)
- System Memory: 4x 2GB GSKILL Ripjaws DDR3 1600MHz (7-8-7-24)
- Processor: Intel Core i5-750 (OC @ 4.0 GHz)
- CPU Cooler: Prolimatech Megahalems (Delta AFB1212SHE PWM Fan)
- Video: ATI Radeon HD 5770 1GB GDDR5 (Catalyst 8.840.3.0)
- Drive 1: OCZ Agility3 SSD 240GB (AGT3-25SAT3-240G)
- Drive 2: Seagate Momentus-XT Solid State Hybrid Drive ST95005620AS 500GB 7200 RPM 4GB Cache SATA 3Gb/s 2.5″
- Enclosure: CM STORM Scout 2 Gaming Case
- PSU: PC Power and Cooling Silencer 750W Crossfire Edition
- Monitor: Samsung 23″; Widescreen LCD/LED Monitor 1920X1080
- Operating System: Windows 7 Home Premium SP1
- Intel EXPI9301 CT Gigabit Ethernet NIC, x1 PCIe 1.1, 1x CAT5
- Intel E10G42BT, X520-T2, 10Gbps Ethernet NIC, x8 PCIe 2.0, 2x CAT6a
- 50-Foot Category-6 Solid Copper Shielded Twisted Pair Patch Cable
- ASUS PCE-AC66 802.11ac Dual-Band Wireless PCI-E Adapter
Networking Comparison Products
- TRENDnet TEW-673GRU Dual Band N300 Wireless Router
- Linksys EA4500 Dual Band N450 Wireless Router
There is a constant and random potential for Wi-Fi signal degradation in a typical home or office environment. A drop in signal strength and the maximum available data rate usually occurs at locations that are distant from the source, or where there are barriers between the source and receiver. I performed all testing with the router(s) in the same location, and moved the PC and Wi-Fi adapter to three different locations in the house. Yes, I dragged a PC and monitor around the house, because it has the highest potential performance for these benchmarks and the Wi-Fi adapter I used is one of the best available devices on the market, right now. The first test location was in the same room, at a distance of 10′ with a clear line of sight between the router and adapter antenna arrays. The second location was 20′ away in an adjacent room, with back-to-back closets directly between the two devices. I don’t think the clothes had as much impact as the two sets of tightly stacked wire hangers did. The third location was with the PC and adapter downstairs, 25′ away from the router in its same location upstairs. The kitchen, which probably has more metal in it than any room in the house, was between the router and the network adapter. In all cases, I used the signal strength display in the adapter software to optimize the location and orientation of the antennas. This is exactly what you should do when setting up any wireless device, taking care to avoid obstructions and reflecting surfaces that might degrade the signal. Signal strength and stability are both important, when optimizing antenna locations. Of course, all this ‘care and feeding’ of your Wi-Fi device goes out the window if you’re using mobile hardware.
The encryption level was set to WPA2-PSK during the entire testing process. You get better throughput without encryption, but most people understand the need for encryption and they use it, at least if my neighborhood is anything to go by. I used the ASUS PCE-AC66 802.11ac Dual-Band Wireless PCI-E Adapter for all tests, in order to focus on the differences in router performance. It proved to be a very sensitive device, pulling in many more local networks than I typically see on my phone, tablet, or notebook. It’s also the only adapter on the market right now that is fully compatible with the full 3×3 802.11ac implementation on the RT-AC66U router. I concentrated my testing on 802.11n and 802.11ac in the 5GHz band, since there is not much point in proving that the newest hardware can perform with an outdated protocol. I know the 802.11b/g standard often allows for better wall penetration, because of the lower frequency that it gets used with, 2.4GHz, v. 5GHz, but most Wi-Fi users are moving to the 5GHz band to avoid the congestion on the older frequency band.
The ASUS RT-AC66U router shipped out with firmware v18.104.22.168.270, which is the latest released version currently available on the ASUS support web page. There was a beta version available that was newer, but I did not test with it.