NTttcp and NetCPS network test tools

From time to time I need to test the network connection between two windows machines and need to get some kind of report. One other thing important when testing is to define some basic parameters for a test scenario e.g. amount of data for transfer, duration of testing etc. In this type of tests, Windows 7 or Windows 2008 network connection information/diagnostics doesn't help me a lot (although Windows Performance could help). I searched through the internet for a small application which doesn't need any installation and can be run on the most windows systems. I liked two tools.
One is NTttcp which is a multithreaded, asynchronous application that sends and receives data between two or more endpoints and reports the network performance for the duration of the transfer. It is essentially a Winsock-based port of the ttcp tool that measures networking performance in terms of bytes transferred per second and CPU cycles per byte. Because it can be difficult to diagnose a system's overall performance without dividing the system into smaller subsystems, NTttcp allows users to narrow the focus of their testing and investigation to just the networking subsystem.
One other tool I found useful and easy to use was NetCPS which is similar to NTttcps but has a simpler report which is ok in most cases.

Here are sample reports from these two tools:

For NTttcp report looks like this:

C:\Program Files (x86)\Microsoft Corporation\NT Testing TCP Tool>NTttcpr -m 1,0,

192.168.13.29 -a 6 -fr

Copyright Version 3.0

Network activity progressing...

Thread Realtime(s) Throughput(KB/s) Throughput(Mbit/s) Avg Bytes per Completion

====== =========== ================ ================== ========================

0 127.312 10542.426 84.339 52066.773

Total Bytes(MEG) Realtime(s) Average Frame Size Total Throughput(Mbit/s)

================ =========== ================== ========================

1342.177280 127.312 1258.575 84.339

Total Buffers Throughput(Buffers/s) Pkts(recv/intr) Intr(count/s) Cycles/Byte

============= ===================== =============== ============= ===========

20480.000 160.865 1 4372.05 63.6

Packets Sent Packets Received Total Retransmits Total Errors Avg. CPU %

============ ================ ================= ============ ==========

581770 1066426 12 0 18.67

For NetCPS report is a bit simpler:

C:\Documents and Settings\divankovic\My Documents\Downloads\NetCPS>netcps -serve
r -m1000
NetCPS 1.0 - Entering server mode. Press ^C to quit
Waiting for new connection...
Client connected from 192.168.0.65
---> CPS 6232910.00 KPS: 6086.83 MPS: 5.94
Avrg CPS 5769014.00 KPS: 5633.80 MPS: 5.50
Peek CPS 6642190.00 KPS: 6486.51 MPS: 6.33
Client disconnected. 104857600 Kb transferred in 18.18 seconds.

Here is NTttcp tool and here is NetCPS tools.

As a administrator in enterprise environment, there are few basics which I unfortunately tend to forget because I do work most of the time in GUI. When I come to new environment where I need to know the infrastructure - as a client computer in domain, you should run netdom command in order to find out

a. list of domain controllers in domain
b. list of fsmo roles distributed between domain controllers
c. you can even join computers in the domain, manage computer accounts etc.

For example a. - you you should run command:

C:\Users\damirivankovic>netdom query dc
List of domain controllers with accounts in the domain:

HYPERV01
HYPERV02
The command completed successfully.

For example b. - you should run command:
C:\Users\damirivankovic>netdom query fsmo

Schema master HYPERV01.domain.local
Domain naming master HYPERV01.domain.local
PDC HYPERV01.domain.local
RID pool manager HYPERV01.domain.local
Infrastructure master HYPERV01.domain.local
The command completed successfully.

For c. and others - please consult the result of command netdom /?

Service status check or delayed service start

During the years of administering different versions of Windows - I stumbled upon a problem where one windows/application service couldn't start because there was some prerequsites missing. Usually, one service is dependent on the other - for example: I had several cases where I for some reason need to reset the server and when server starts, some service fails because DB service isn't started yet. In these situations you usually end up in problems where application isn't working and you need to start service manually in order to fix the problem. This is why I find plain batch script very useful and it works for all windows versions. Below I wrote a script for postponing service startup for 5 minutes. Since, windows generaly doesn't have this tool installed - you should download tool sleep.exe from Microsoft web site, wich is part of Windows 2003 Resource Kit. Download resource kit and install it or unpack it and copy sleep.exe to some folder in system path.

Script goes something like:

@echo off

ECHO Delaying application service start for x minutes

sleep.exe 300

NET START "name of the application service"

Now, configure problematic service for a manual start and save this batch script and schedule it for a start at system boot. This will function as a delayed start of the service. This is feature already built into Windows 2008, but it doesn't exist in other windows versions.

To fine-tune a bit more, you should create this batch script and it's shortcut to configure this to start in minimized window so it doesn't obstruct users.

To go a little further, in cases where dependent services exist on the same computer, you should do the service status checking before doing some action of stopping or starting the service. For example - to check if one service is running you should include this in your script:

@echo off

ECHO Checking service status

NET START | FIND "name of the service1" > nul

IF errorlevel 1 GOTO :not_running

IF errorlevel 0 GOTO :is_running

:is_running

NET STOP "name of the service2"

:not_running

sleep.exe 300

NET START "name of the service2"

This script will do the service status checking and if service1 is running it will stop the service2 or if service1 is stopped it will do the delayed start of the service2. I believe you now understand what could you do with this.

Hope this helps. :)

Windows 2003 time synchronization - to external time source

Although this isn't in my primary focus, time synchronization with external time source is very important. This is especially important for Windows domain environments, but also for larger systems in workgroups. I searched everywhere for this simple information - how to sync clock of the Windows computer(s) in workgroup to some external time source i.e. ntp servers?

I know that this looks simple - but if you really tried to find relevant information for this case, you will usually start at Microsoft Technet or Windows command line (w32tm tool) and eventually end up on user forums browsing through articles or posting questions why this doesn't work. There are several way to do this - one is using Windows Time and Date from control panel (or by doubleclicking on clock on the taskbar). Second way is by changing some setting in registry and using command line and the third way is by using command line. This third way is what I will show you. I find that using command line is best for me - since I can control and validate synchronization process. Benefits of using scripts are:

• Control the start process
• Schedule the sync job
• Log synchronization results in txt file
  
• Log synchronization results in Event Viewer (System)

What you will need?

Of course, I assume that you are using Windows 2003 in workgroup environment. You will need Internet connection in order to access time servers and you need to choose the time server you like. Usually, this is NTP server, time server in your country or at least in your time zone. You can search on the internet for available time servers. Last but not the least - if you are connected to some kind of corporate local network with firewall - you need to ensure that UDP port 123 is open for traffic originating from internal network to the internet.

Anyway, since I live in Croatia I will be using one of the local NTP/time servers "zg1.ntp.carnet.hr". Before running sync job, let's check/notice the time difference you have now between your computer clock and correct time showed by one of the time servers in your area. Now, create the batch file with following lines:

w32tm /config /manualpeerlist:zg1.ntp.carnet.hr,0x8 /syncfromflags:MANUAL /reliable:yes
w32tm /config /update
net stop w32time
net start w32time
w32tm /resync /rediscover

You should save this file on your disk and test if this script work for your computer.
After you run this script - you should notice this change i.e. that the time on your computer is now correct.

Hope this helps! :)

Uploading large files to Sharepoint

I've stumbled upon a problem when I tried to upload 200MB file into SharePoint Video library. Error message was: "The form submission cannot be processed because it exceeded the maximum length allowed by the Web administrator. Please resubmit the form with less data."

By the book - you should change Maximum Upload Size field of desired web application. This is how you do it.

By default SharePoint allows a maximum file size of 50MB that can be uploaded into any document library. This might be too much as you might not want users downloading 50MB files over your LAN/WAN. Anyway, to change the maximum file size

1. Goto SharePoint Central Administration > Select Application Management
2. Under SharePoint Web Application Management - select Web application general settings
3. Select your web application (usually you would select app. running on port 80)
4. Change the value of the Maximum Upload Size in MB's.

This should work but it didn't work for me on my default/standard MOSS 2007 installation. Read on to find out what else is needed if this doesn't help you.

Changing Maximum Upload Size to 200MB wasn't enough for me although the file I was uploading was 166MB!? What now?

This was happening because the web.config of the application was blocking the change in Sharepoint Central Management. I had to change the web.config of the defualt application (working on port 80) and the web.config located in 12Hive folder which is located in C:\Program Files\Common Files\Microsoft Shared\Web server extensions\12\TEMPLATE\LAYOUTS.

I've searched both web.configs for the "" code. After I found this line in web.config I've done two things:

1. I've added executionTimeout="999999" code for timeout time to support longer uploads due to support for bigger file size.
2. I've changed the default code maxRequestLength="51200" to support larger files.

Before change, the code was:

After changing this line, the code was:

After I changed this I was able to upload my 166MB video to Sharepoint.

Hope this was helpfull.

Auto-numbering InfoPath forms when they are submitted to a SharePoint form library

Problem: You want to automatically assign a number to InfoPath form whenever a new form is submitted into form library?

I know this sounds strange - but this isn't easy thing to setup in form library - as you will see in few moments from the text below.

Solution: Use rules, formulas, and the max() function to generate incremental numbers that can be assigned to InfoPath forms when they are submitted to a form library.

You can accomplish this functionality as follows:
1. In Sharepoint, create a form library to submit your forms to
2. In InfoPath, create a new InfoPath form template
3. Add a Text Box control to the form template and name it autonumber
4. On the Tools menu click Data Connections
5. On the Data Connections dialog box, click Add, and create a Receive data connection to the Sharepoint library that you created in step 1. In step 5 of the Data connection wizard, uncheck all field and then check the ID field. In the last steop od the data connection wizard, uncheck Automatically retrieve data when form is opened, name the data connection RetrieveIDs and click Finish.
6. On the Data Connections dialog box, click Add, and create a Submit data connection to the Sharepoint library that you created in step 1. In step 3 of the data connection wizard, click the formula button behind the File name field.
7. On the Insert formula dialog box, type max( and then click Insert Field or Group.
8. On the Select a field or Group dialog box, select RetrieveIDs from the Data Source drop-down list box, expand all the nodex, click ID, and then click OK
9. On the Insert Formula dialog box, type ) + 1 and click OK.
10. On the Data Connection Wizard dialog box, the expression in the File Name field should look like:
max (@ID) + 1
11. On the Data Connection Wizard dialog box, click Next, name the data connection Main submit, and then click Finish
12. On the Data Connections dialog box, click Close
13. On the Tools menu, click Submit Options
14. On the Submit Options dialog box, check the Allow users to submit this form checkbox, select the Perform custom actions using rules option, click Advanced, select Close the form from the After submit drop-down list box, and then click Rules
15. On the Rules dialog box, click Add
16. On the Rule dialog box, add an Action that says:
Query using data connection: RetrieveIDs
17. On the Rule dialog box, add a second Action that says
Set field's value: autonumber = max(@ID) + 1

Note: @ID is selected from the RetrieveIDs secondary data source.

18. On the Rule dialog box, add a third Action that says:
Submit using data connection: Main submit
19. Click OK when closing all dialog boxes that are open.
20. Publish the form template to the library you created in step 1.

That should do it. Please try.

Sharepoint Stress Test

In search for the way to stress test my Sharepoint, I stumbled upon Microsoft Application Center Test (Microsoft ACT). ACT is a part of Visual Studio .NET 2003 so it isn't available as a standalone application. I wouldn't call ACT the ultimate stress test for SharePoint, but this is valuable tool to mention. The point of stress testing SharePoint is to see or at least get some kind of feeling how much users can your SharePoint handle. This is valuable information you need to know if you are building web pages, portals or intranets. The important thing is - don't forget to stress test your web site BEFORE publishing it (making it available for your audience) - or you could end up in dead-end, having your web site experience to be bad and sluggish.

As a starting point, I would suggest that you find some of your past projects (web sites), use any of existing intranet websites - something that you know that is working correctly and that it's performance was not an issue before. Then I would sugesst you stress test that website and see what numbers you will get as a result of stress test.



Example above shows very low RPS (Requests Per Second) number. This is because the network link was a bottleneck in this test scenario. Of course, when stress testing system - you need to setup test environment where there are no bottlenecks between you (testing system) and SharePoint frontend (tested system) - otherwise you will see irrelevant numbers in ACT.

Testing system
Testing system is the machine where you install ACT (Visual Studio) and use it as client, to connect to the website you wish to stress test. I suggest you place your testing system on the same network switch where your SharePoint platform is. Use gigabit network cards if available(100Mbps is minimum). Install ACT on any P4 or multi-core CPU system with at least 2GB of RAM memory where at least 512MB is available/free to operate ACT. I was using my DuoCore laptop as a testing system.

Tested system
Tested system is the machine (solution, platform or system) where you host your website. If you are testing more complex (multi tier, multi-machine) platform like SharePoint, where you have few servers, two or more in loadbalanced frontend and DB server in backend - then you test frontend web servers. I don't want to talk about the system design (that is whole new topic) but you have to keep in mind that bad system design will have drastic influence on this test results. For example, slow disk performance on DB server could result in sluggish website performance on web frontend and overall result would be poor performance of the whole website (showing low RPS numbers). This is why it is important to have good basis for each part of the solution.
Anyway, I assume that servers are ok, network links between servers (if any) are ok, there are resources (RAM, CPU) available. As previously said - tested system is accessed through frontend web servers and this is what we need to test.

Goal
By testing performance of frontend web server we are testing the whole solution. The goal of stress test is to stress frontend web servers with enough load until we reach point where frontend(s) CPU usage is constantly reaching maximum numbers or even better - when we reach CPU usage of 90%. 90% of CPU usage should be our goal here because after 90% of CPU usage - system is being less responsive and it is tending to loosing packets and we do not want to have that. When we reach 90% of CPU usage on frontend, we should stop adding load on the website and write down our RPS numbers from ACT. When we find out our RPS numbers - we can compare these to other systems or if we find these numbers to be lower then we expect them to be - we could upgrade existing server or add another web server in load-balanced frontend and run the stress test again. The point in adding additional web servers into load-balanced frontend is to reach the point where we are unable to reach 90% of usage on web server becuase another component of the system is performance bottleneck (e.g. disk or CPU on DB server)