Wednesday, March 31, 2010

Sharepoint Stress Test

In search for the way to stress test my Sharepoint, I stumbled upon Microsoft Application Center Test (Microsoft ACT). ACT is a part of Visual Studio .NET 2003 so it isn't available as a standalone application. I wouldn't call ACT the ultimate stress test for SharePoint, but this is valuable tool to mention. The point of stress testing SharePoint is to see or at least get some kind of feeling how much users can your SharePoint handle. This is valuable information you need to know if you are building web pages, portals or intranets. The important thing is - don't forget to stress test your web site BEFORE publishing it (making it available for your audience) - or you could end up in dead-end, having your web site experience to be bad and sluggish.

As a starting point, I would suggest that you find some of your past projects (web sites), use any of existing intranet websites - something that you know that is working correctly and that it's performance was not an issue before. Then I would sugesst you stress test that website and see what numbers you will get as a result of stress test.



Example above shows very low RPS (Requests Per Second) number. This is because the network link was a bottleneck in this test scenario. Of course, when stress testing system - you need to setup test environment where there are no bottlenecks between you (testing system) and SharePoint frontend (tested system) - otherwise you will see irrelevant numbers in ACT.

Testing system
Testing system is the machine where you install ACT (Visual Studio) and use it as client, to connect to the website you wish to stress test. I suggest you place your testing system on the same network switch where your SharePoint platform is. Use gigabit network cards if available(100Mbps is minimum). Install ACT on any P4 or multi-core CPU system with at least 2GB of RAM memory where at least 512MB is available/free to operate ACT. I was using my DuoCore laptop as a testing system.

Tested system
Tested system is the machine (solution, platform or system) where you host your website. If you are testing more complex (multi tier, multi-machine) platform like SharePoint, where you have few servers, two or more in loadbalanced frontend and DB server in backend - then you test frontend web servers. I don't want to talk about the system design (that is whole new topic) but you have to keep in mind that bad system design will have drastic influence on this test results. For example, slow disk performance on DB server could result in sluggish website performance on web frontend and overall result would be poor performance of the whole website (showing low RPS numbers). This is why it is important to have good basis for each part of the solution.
Anyway, I assume that servers are ok, network links between servers (if any) are ok, there are resources (RAM, CPU) available. As previously said - tested system is accessed through frontend web servers and this is what we need to test.

Goal
By testing performance of frontend web server we are testing the whole solution. The goal of stress test is to stress frontend web servers with enough load until we reach point where frontend(s) CPU usage is constantly reaching maximum numbers or even better - when we reach CPU usage of 90%. 90% of CPU usage should be our goal here because after 90% of CPU usage - system is being less responsive and it is tending to loosing packets and we do not want to have that. When we reach 90% of CPU usage on frontend, we should stop adding load on the website and write down our RPS numbers from ACT. When we find out our RPS numbers - we can compare these to other systems or if we find these numbers to be lower then we expect them to be - we could upgrade existing server or add another web server in load-balanced frontend and run the stress test again. The point in adding additional web servers into load-balanced frontend is to reach the point where we are unable to reach 90% of usage on web server becuase another component of the system is performance bottleneck (e.g. disk or CPU on DB server)


Monday, March 29, 2010

The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist

I wanted to share something with those of You who use SharePoint MOSS 2007. I had one interesting problem with SharePoint after doing some customization of menus/navigation in SharePoint.


SharePoint navigation is done either by using Quick launch menus or by using Top Navigational menu. Both menus are standard out-of-the box menus in SharePoint. Since SharePoint is content management system, it handles those menus in some specific way. For example - after you add some lists to your site - SharePoint automatically add link to this list on the quick launch menu. This is cool feature, but what if you don't like this? Well, in this case - you need to substitute menu with xml. Basically, this substitution consists of several steps. First, you need to define CustomXmlContentMapProvider in web.config of the website you wish to edit. Second, you need to create the xml file with menu structure and place it somewhere within web site folder so it is accessible to the web application. Third, edit the default.master page in Sharepoint and define the DataSourceID and SiteMapDataSource. After you are done with this - you will have your own totally customizable menu structure of Sharepoint site.


Now, this is where tricky part starts. When I done this - everything worked fine - at least I thought so. From time to time, it looked totally random - people would receive "The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist". Web.Sitemap is actually the xml file I made for menu structure. So, this error means that "XmlSiteMapProvider" couldn't find Web.sitemap. But thisisn't true because this file is located in _app_bin/Web.sitemap folder. Whenever I tried to open Sharepoint web site - everything opened succesfully. I would usually then double-check with the person that reported this error and usually then everything would be working. I was confused at that time. After some time, I noticed a pattern here - usually this happens early in the morning and usually after I open the website - error dissapears i.e. Sharepoint is then working for everyone (at least till next morning).

After I evaluated situation a little bit - I concluded that this could be related to permissions somehow and it happens after some period of inactivity of the website. Since, I am Sharepoint administrator, after I open the website - everything works so I concluded that after this Web.sitemap is loaded first time, everything is working because this Web.sitemap is somehow cached and is available for everyone until next period of long inactivity.

I then checked to see what permissions are effective on file Web.sitemap in Inetpub folder of Sharepoint website. I noticed that, only Administrators have access to web.sitemap. Since, I am Sharepoint administrator, I can access this website, but no domain users can access this file -> which could lead to this error if domain users try to access this file before any of the administrators. I then added read permissions to domain users group on file Web.sitemap and then - problem dissapeared! :=)

I have monitored Sharepoint for weeks after but still no error.

I hope this article will help you so you know that you need to give read permissions to domain users in domain or to Everyone group if this site is accessible from the internet.

Best regards,
Damir.




Below is the error I would get from time to time, randomly.
--

Server Error in '/' Application.


The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.


Stack Trace:

[InvalidOperationException: The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist.]

System.Web.XmlSiteMapProvider.CheckSiteMapFileExists() +2603440

System.Web.XmlSiteMapProvider.GetConfigDocument() +226

System.Web.XmlSiteMapProvider.BuildSiteMap() +58

Microsoft.SharePoint.Navigation.SPXmlContentMapProvider.BuildSiteMap() +53

System.Web.XmlSiteMapProvider.get_RootNode() +15

System.Web.UI.WebControls.SiteMapDataSource.GetNodes() +229

System.Web.UI.WebControls.SiteMapDataSource.GetTreeView(String viewPath) +35

System.Web.UI.WebControls.SiteMapDataSource.GetHierarchicalView(String viewPath) +32

System.Web.UI.HierarchicalDataSourceControl.System.Web.UI.IHierarchicalDataSource.GetHierarchicalView(String viewPath) +10

System.Web.UI.WebControls.HierarchicalDataBoundControl.GetData(String viewPath) +25

System.Web.UI.WebControls.Menu.DataBindItem(MenuItem item) +56

System.Web.UI.WebControls.Menu.PerformDataBinding() +50

System.Web.UI.WebControls.HierarchicalDataBoundControl.PerformSelect() +85

System.Web.UI.WebControls.BaseDataBoundControl.DataBind() +73

System.Web.UI.WebControls.Menu.DataBind() +4

System.Web.UI.WebControls.BaseDataBoundControl.EnsureDataBound() +82

System.Web.UI.WebControls.Menu.EnsureDataBound() +40

System.Web.UI.WebControls.BaseDataBoundControl.OnPreRender(EventArgs e) +22

System.Web.UI.WebControls.Menu.OnPreRender(EventArgs e, Boolean registerScript) +15

System.Web.UI.WebControls.Menu.OnPreRender(EventArgs e) +25

Microsoft.SharePoint.WebControls.AspMenu.OnPreRender(EventArgs e) +142

System.Web.UI.Control.PreRenderRecursiveInternal() +80

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +842

Wednesday, March 10, 2010

What every SharePoint administrator needs to know about Alternate Access Mappings?

Taken from Blogs.msdn.com post at http://blogs.msdn.com/sharepoint/archive/2007/03/06/what-every-sharepoint-administrator-needs-to-know-about-alternate-access-mappings-part-1.aspx

--
Hi folks, this is Troy Starr again from the Windows SharePoint Services Test team. Today I'd like to talk about one of the most powerful, but often one of the least understood, features in Windows SharePoint Services 3.0 and Microsoft Office SharePoint Server 2007. That feature is called Alternate Access Mappings. Around here, we just call it "AAM" for short.

At the most basic level, AAM tells SharePoint how to map web requests (for example, browsing to the homepage of a SharePoint site) to the correct web application and site so that SharePoint can serve the correct content back to you. It then tells SharePoint what URL the users should be taken to as they interact with SharePoint.

Seems simple enough, doesn’t it? Those of you who are familiar with developing web applications in Internet Information Services may be wondering right now why we need such a feature since IIS can tell you what the URL of an incoming web request is. The major reason we need this is that there are common Internet deployment scenarios where the URL of a web request received by IIS is not the same URL that the end user entered. These are most common in reverse proxy publishing and load balancing scenarios.

How is this possible? Let's consider a reverse proxy scenario. A reverse proxy is a device that sits between end users and your web server. All requests to your web server are first received by the reverse proxy device, and if those requests pass the proxy's security filtering, the proxy will forward the requests on to your web server. Reverse proxies can perform advanced functionality such as receiving a web request over the Internet via SSL (HTTPS), but forward the request to the your server via HTTP. This is referred to as off-box SSL termination. They can also forward the request to a different port number than it was originally received on and can even change the HTTP Host header field. If SharePoint were base its links off of the URL of the request it received, the links that end users click on could be the incorrect "internal" URL rather than the correct "public" URL.

SharePoint is compatible with a variety of reverse proxy servers, but for this example we'll take a look at a publishing rule from Microsoft's reverse proxy software - Internet Security and Acceleration Server 2006. ISA Server 2006 includes a SharePoint publishing wizard that walks you through creating a publishing rule for SharePoint. Once the rule is created, you can modify it at any time. (The following images show a slightly modified publishing rule where the "Forward the original host header" option is turned off to help demonstrate the flexibility of AAM. If we left the "Forward the original host header" option turned on, the public hostname would also serve as the internal hostname when configuring AAM.) The first two dialogs show the "listener" and "public name" properties of the rule, which define what URL users will use to access your SharePoint site. Remember that this URL is really the URL of your reverse proxy server, which will forward the request to your SharePoint server.






The end user's URL is comprised of the public protocol, the public hostname, and the public port number.

Public Protocol
+
"://"
+
Public Hostname
+
":"
+
Public Port Number
=
Public URL

HTTPS
www.contoso.com
443
https://www.contoso.com


The next two dialogs show the "to" and "bridging" properties of the rule, which define what URL the reverse proxy server will use to forward the request to your SharePoint server.





The SharePoint server's URL is comprised of the internal protocol, the internal hostname, and the internal port number.

Internal Protocol
+
"://"
+
Internal Hostname
+
":"
+
Internal Port Number
=
Internal URL

HTTP
sharepoint.dmz.contoso.com
80
http://sharepoint.dmz.contoso.com




Great - we've properly set up this reverse proxy server to receive web requests from end users at https://www.contoso.com and forward them to your SharePoint server at http://sharepoint.dmz.contoso.com. We're halfway there! The next step is to configure your SharePoint web application and AAM to match the publishing rule above. To do this, we'll extend an existing web application to an additional IIS Web site just for your reverse proxy's publishing rule. Note that you're also able to create a new web application from scratch for this publishing rule - the fields you'll need to fill out are the same in either case.

Browse to your WSS 3.0 Central Administration site and click on the Application Management tab. Next, click "Create or extend Web application" and then click "Extend an existing Web application." Select the web application that you wish to use, and then fill out the port, host header, and SSL fields based on the internal URL properties that we defined above. In the URL field, enter the public URL that we defined above. Finally, you'll want to select an AAM Zone to assign this extension of your web application to. There are a maximum of 5 zones available in each web application. We'll use the Internet zone in this example, but you're free to use any available zone. All of the zones provide the same functionality, although the Default zone will always be used for certain features such as sending administrative e-mail messages to site collection owners. When you're finished, click OK to create the IIS Web site.



Next, you'll want to verify that your public URL was created correctly in AAM and then add your internal URL. Unless your internal URL is the same as your public URL, this is an extra step that you must perform manually. To do this, browse to your WSS 3.0 Central Administration site and click on the Operations tab. Next, click "Alternate access mappings." Click the Alternate Access Mappings selector drop-down and select the web application that you're publishing through your reverse proxy server. You should now see the AAM URLs assigned to your web application.




As you can see, the public URL from the reverse proxy publishing rule has been assigned to your web application's Internet zone. The final touch is to add the internal URL from the reverse proxy publishing rule to your web application's Internet zone. To do this, click "Add Internal URLs" in the AAM toolbar, type in the internal URL, and select the same zone that you used for the public URL. In this case, that was the Internet zone. When you're finished, click Save. You should now see the additional URL is assigned to your web application, in the same zone as the public URL of your reverse proxy publishing rule.




All done! Now, when a user browses to https://www.contoso.com, the web request will be received by the proxy server and forwarded to http://sharepoint.dmz.contoso.com. SharePoint will then receive the web request, see that the URL of the request is http://sharepoint.dmz.contoso.com, find that this URL is assigned to the Contoso web application, and return the content from that web application. In addition, because the http://sharepoint.dmz.contoso.com URL is assigned to the Internet zone, SharePoint will also generate links on the pages using the public URL for that zone - https://www.contoso.com. This ensures that end users are taken to the proper URL when clicking on links on the page.

Load balancers work similarly, especially if they overwrite the end user's original URL with the URL of the individual web server that the request is being load balanced to. To address this, just add each individual web server's URL to AAM as an internal URL and associate them all to the same zone as end user's public URL.

I hope that this introduction to Alternate Access Mappings was helpful to you. Please feel free to post comments to this blog entry with any questions you may have about AAM. I will be posting another blog entry soon covering common AAM mistakes and how to avoid them.



Troy Starr

How to backup VMWare Virtual Machines?


As you may already know - backing up virtual machines isn't a walk in the park. Actually, it is very much similar to usual backup process of any physical machine. However, VMWare has a nice tool called "VMware Consolidated Backup" which can do even online backups of virtual machines, without disrupting users and applications. Of course this tool isn't free. So, we are back at the begining of this story - how to backup virtual machines.

There are few ways to do manual backup of virtual machines which can then bi scripted into a semi-automatic backup or automatic backup if you whish. The down side of this approach is that you have downtime of virtual machine when doing backup, but in most cases I find this acceptable.

We are using VMWare server 2.0 platform for our virtual machines. There are few requirements in order to do this. First, you need to familiarize yourself with vmrun command line tool. Vmrun is used for managing virtual machines from command line. This and other Vmware tools are by default located in c:\Program Files\VMWare\VMware Server\.

This is what I use for "soft" stopping virtual machine SPWFE1.domain.local located at E_DRIVE datastore of VMWare Server named VM.domain.local. I am using local administrator user context for running this command i.e. user:admin with password:wordxxxx

From command prompt run: "vmrun -T server -h https://vm.domain.local:8333/sdk -u admin -p wordxxxx stop "[E_DRIVE] spwfe1\spwfe1.domain.local.vmx" soft"

This command, shutsdown virtual machine spwfe1.domain.local after ending all processes.

Now, we need to go to next step, which is copying the virtual machine to another location i.e. making backup of the virtual machine. In order to do that - I use xcopy command to copy folder content to another location.

After copying is done, we can start the virtual machine by running command "vmrun -T server -h https://vm.domain.local:8333/sdk -u admin -p wordxxxx start "[E_DRIVE] spwfe1\spwfe1.domain.local.vmx"

The final thing, in order to automate this, you should organize stop, copy and startup scripts into different scripts and use either windows AT command or Scheduled Tasks to schedule starting of these scripts.

Sharepoint system architecture


When thinking about sharepoint architecture - you got to first think about the platform architecture and system design. Nowdays, virtualization technology is something we all need to think about when planning system design of any kind. That goes also for sharepoint architecture.

Sharepoint, as we all know is Microsoft's content management system and collaboration tool for groupwares. As such, it is used very often as company's Intranet or Internet presence platform. Since Sharepoint allows you to distribute it's components through many hardware boxes in order to load balance requests - it can be used in so called multi-layerd architecture also known as multi-tier architecture. In multi-tier architecture - Sharepoint can be installed in load-balancing and fail-over environment to provide better performance and high availability.

So, ideal Sharepoint architecture is comprised of multiple hardware boxes serving in multi-tier architecture. Virtualization technology fits well in this multi-box multi-tier architecture. There is only one rule you need to know about - virtualization of database servers is not something you should do or use in production environment. The main reason is because DBs often do a lot I/O disk activity which doesn't perform so well in virtual environment. On the other hand - it is ok to have virtualized DB server in testing environment.

Sharepoint backup error: Object xyz_DB failed in event OnBackup

I started having problems with Sharepoint backup some time ago. The problems were that I would get an error message "Object SharedServices1_Search_DB failed in event OnBackup" and similar started showing in Central Administration backup when I moved my backup location from Sharepoint server to UNC network location. I tried many things but - I even tried to login with Sharepoint service account and do the backup but it didn't work - even though this same account "spservice" had "Full control" on this network share folder. So, what eventually did help me was to (for testing purposes only) change NTFS permissions on network share where my backup was done. I gave Everyone and Anonymous group "Full control" permissions on my \\fileserver\StsadmBackup folder. After I did that - backup was done succesfully! :)

Anyway, at least now I know that everything is OK with my Sharepoint frontend and DB servers. It was permissions that were the issue. I will probably move backup folder back to frontend machine or I will leave UNC path on network share but really need to get rid of anonymous and everyone groups.

Regards,
Damir.