Tuesday, September 14, 2010

Uploading large files to Sharepoint



I've stumbled upon a problem when I tried to upload 200MB file into SharePoint Video library. Error message was: "The form submission cannot be processed because it exceeded the maximum length allowed by the Web administrator. Please resubmit the form with less data."

By the book - you should change Maximum Upload Size field of desired web application. This is how you do it.

By default SharePoint allows a maximum file size of 50MB that can be uploaded into any document library. This might be too much as you might not want users downloading 50MB files over your LAN/WAN. Anyway, to change the maximum file size

1. Goto SharePoint Central Administration > Select Application Management
2. Under SharePoint Web Application Management - select Web application general settings
3. Select your web application (usually you would select app. running on port 80)
4. Change the value of the Maximum Upload Size in MB's.

This should work but it didn't work for me on my default/standard MOSS 2007 installation. Read on to find out what else is needed if this doesn't help you.

Changing Maximum Upload Size to 200MB wasn't enough for me although the file I was uploading was 166MB!? What now?

This was happening because the web.config of the application was blocking the change in Sharepoint Central Management. I had to change the web.config of the defualt application (working on port 80) and the web.config located in 12Hive folder which is located in C:\Program Files\Common Files\Microsoft Shared\Web server extensions\12\TEMPLATE\LAYOUTS.

I've searched both web.configs for the "" code. After I found this line in web.config I've done two things:

1. I've added executionTimeout="999999" code for timeout time to support longer uploads due to support for bigger file size.
2. I've changed the default code maxRequestLength="51200" to support larger files.

Before change, the code was:





After changing this line, the code was:




After I changed this I was able to upload my 166MB video to Sharepoint.

Hope this was helpfull.

Saturday, August 7, 2010

Auto-numbering InfoPath forms when they are submitted to a SharePoint form library

Problem: You want to automatically assign a number to InfoPath form whenever a new form is submitted into form library?

I know this sounds strange - but this isn't easy thing to setup in form library - as you will see in few moments from the text below.

Solution: Use rules, formulas, and the max() function to generate incremental numbers that can be assigned to InfoPath forms when they are submitted to a form library.

You can accomplish this functionality as follows:
1. In Sharepoint, create a form library to submit your forms to
2. In InfoPath, create a new InfoPath form template
3. Add a Text Box control to the form template and name it autonumber
4. On the Tools menu click Data Connections
5. On the Data Connections dialog box, click Add, and create a Receive data connection to the Sharepoint library that you created in step 1. In step 5 of the Data connection wizard, uncheck all field and then check the ID field. In the last steop od the data connection wizard, uncheck Automatically retrieve data when form is opened, name the data connection RetrieveIDs and click Finish.
6. On the Data Connections dialog box, click Add, and create a Submit data connection to the Sharepoint library that you created in step 1. In step 3 of the data connection wizard, click the formula button behind the File name field.
7. On the Insert formula dialog box, type max( and then click Insert Field or Group.
8. On the Select a field or Group dialog box, select RetrieveIDs from the Data Source drop-down list box, expand all the nodex, click ID, and then click OK
9. On the Insert Formula dialog box, type ) + 1 and click OK.
10. On the Data Connection Wizard dialog box, the expression in the File Name field should look like:
max (@ID) + 1
11. On the Data Connection Wizard dialog box, click Next, name the data connection Main submit, and then click Finish
12. On the Data Connections dialog box, click Close
13. On the Tools menu, click Submit Options
14. On the Submit Options dialog box, check the Allow users to submit this form checkbox, select the Perform custom actions using rules option, click Advanced, select Close the form from the After submit drop-down list box, and then click Rules
15. On the Rules dialog box, click Add
16. On the Rule dialog box, add an Action that says:
Query using data connection: RetrieveIDs
17. On the Rule dialog box, add a second Action that says
Set field's value: autonumber = max(@ID) + 1

Note: @ID is selected from the RetrieveIDs secondary data source.

18. On the Rule dialog box, add a third Action that says:
Submit using data connection: Main submit
19. Click OK when closing all dialog boxes that are open.
20. Publish the form template to the library you created in step 1.

That should do it. Please try.

Wednesday, March 31, 2010

Sharepoint Stress Test

In search for the way to stress test my Sharepoint, I stumbled upon Microsoft Application Center Test (Microsoft ACT). ACT is a part of Visual Studio .NET 2003 so it isn't available as a standalone application. I wouldn't call ACT the ultimate stress test for SharePoint, but this is valuable tool to mention. The point of stress testing SharePoint is to see or at least get some kind of feeling how much users can your SharePoint handle. This is valuable information you need to know if you are building web pages, portals or intranets. The important thing is - don't forget to stress test your web site BEFORE publishing it (making it available for your audience) - or you could end up in dead-end, having your web site experience to be bad and sluggish.

As a starting point, I would suggest that you find some of your past projects (web sites), use any of existing intranet websites - something that you know that is working correctly and that it's performance was not an issue before. Then I would sugesst you stress test that website and see what numbers you will get as a result of stress test.



Example above shows very low RPS (Requests Per Second) number. This is because the network link was a bottleneck in this test scenario. Of course, when stress testing system - you need to setup test environment where there are no bottlenecks between you (testing system) and SharePoint frontend (tested system) - otherwise you will see irrelevant numbers in ACT.

Testing system
Testing system is the machine where you install ACT (Visual Studio) and use it as client, to connect to the website you wish to stress test. I suggest you place your testing system on the same network switch where your SharePoint platform is. Use gigabit network cards if available(100Mbps is minimum). Install ACT on any P4 or multi-core CPU system with at least 2GB of RAM memory where at least 512MB is available/free to operate ACT. I was using my DuoCore laptop as a testing system.

Tested system
Tested system is the machine (solution, platform or system) where you host your website. If you are testing more complex (multi tier, multi-machine) platform like SharePoint, where you have few servers, two or more in loadbalanced frontend and DB server in backend - then you test frontend web servers. I don't want to talk about the system design (that is whole new topic) but you have to keep in mind that bad system design will have drastic influence on this test results. For example, slow disk performance on DB server could result in sluggish website performance on web frontend and overall result would be poor performance of the whole website (showing low RPS numbers). This is why it is important to have good basis for each part of the solution.
Anyway, I assume that servers are ok, network links between servers (if any) are ok, there are resources (RAM, CPU) available. As previously said - tested system is accessed through frontend web servers and this is what we need to test.

Goal
By testing performance of frontend web server we are testing the whole solution. The goal of stress test is to stress frontend web servers with enough load until we reach point where frontend(s) CPU usage is constantly reaching maximum numbers or even better - when we reach CPU usage of 90%. 90% of CPU usage should be our goal here because after 90% of CPU usage - system is being less responsive and it is tending to loosing packets and we do not want to have that. When we reach 90% of CPU usage on frontend, we should stop adding load on the website and write down our RPS numbers from ACT. When we find out our RPS numbers - we can compare these to other systems or if we find these numbers to be lower then we expect them to be - we could upgrade existing server or add another web server in load-balanced frontend and run the stress test again. The point in adding additional web servers into load-balanced frontend is to reach the point where we are unable to reach 90% of usage on web server becuase another component of the system is performance bottleneck (e.g. disk or CPU on DB server)


Monday, March 29, 2010

The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist

I wanted to share something with those of You who use SharePoint MOSS 2007. I had one interesting problem with SharePoint after doing some customization of menus/navigation in SharePoint.


SharePoint navigation is done either by using Quick launch menus or by using Top Navigational menu. Both menus are standard out-of-the box menus in SharePoint. Since SharePoint is content management system, it handles those menus in some specific way. For example - after you add some lists to your site - SharePoint automatically add link to this list on the quick launch menu. This is cool feature, but what if you don't like this? Well, in this case - you need to substitute menu with xml. Basically, this substitution consists of several steps. First, you need to define CustomXmlContentMapProvider in web.config of the website you wish to edit. Second, you need to create the xml file with menu structure and place it somewhere within web site folder so it is accessible to the web application. Third, edit the default.master page in Sharepoint and define the DataSourceID and SiteMapDataSource. After you are done with this - you will have your own totally customizable menu structure of Sharepoint site.


Now, this is where tricky part starts. When I done this - everything worked fine - at least I thought so. From time to time, it looked totally random - people would receive "The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist". Web.Sitemap is actually the xml file I made for menu structure. So, this error means that "XmlSiteMapProvider" couldn't find Web.sitemap. But thisisn't true because this file is located in _app_bin/Web.sitemap folder. Whenever I tried to open Sharepoint web site - everything opened succesfully. I would usually then double-check with the person that reported this error and usually then everything would be working. I was confused at that time. After some time, I noticed a pattern here - usually this happens early in the morning and usually after I open the website - error dissapears i.e. Sharepoint is then working for everyone (at least till next morning).

After I evaluated situation a little bit - I concluded that this could be related to permissions somehow and it happens after some period of inactivity of the website. Since, I am Sharepoint administrator, after I open the website - everything works so I concluded that after this Web.sitemap is loaded first time, everything is working because this Web.sitemap is somehow cached and is available for everyone until next period of long inactivity.

I then checked to see what permissions are effective on file Web.sitemap in Inetpub folder of Sharepoint website. I noticed that, only Administrators have access to web.sitemap. Since, I am Sharepoint administrator, I can access this website, but no domain users can access this file -> which could lead to this error if domain users try to access this file before any of the administrators. I then added read permissions to domain users group on file Web.sitemap and then - problem dissapeared! :=)

I have monitored Sharepoint for weeks after but still no error.

I hope this article will help you so you know that you need to give read permissions to domain users in domain or to Everyone group if this site is accessible from the internet.

Best regards,
Damir.




Below is the error I would get from time to time, randomly.
--

Server Error in '/' Application.


The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist.

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.InvalidOperationException: The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist.

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.


Stack Trace:

[InvalidOperationException: The file _app_bin/Web.sitemap required by XmlSiteMapProvider does not exist.]

System.Web.XmlSiteMapProvider.CheckSiteMapFileExists() +2603440

System.Web.XmlSiteMapProvider.GetConfigDocument() +226

System.Web.XmlSiteMapProvider.BuildSiteMap() +58

Microsoft.SharePoint.Navigation.SPXmlContentMapProvider.BuildSiteMap() +53

System.Web.XmlSiteMapProvider.get_RootNode() +15

System.Web.UI.WebControls.SiteMapDataSource.GetNodes() +229

System.Web.UI.WebControls.SiteMapDataSource.GetTreeView(String viewPath) +35

System.Web.UI.WebControls.SiteMapDataSource.GetHierarchicalView(String viewPath) +32

System.Web.UI.HierarchicalDataSourceControl.System.Web.UI.IHierarchicalDataSource.GetHierarchicalView(String viewPath) +10

System.Web.UI.WebControls.HierarchicalDataBoundControl.GetData(String viewPath) +25

System.Web.UI.WebControls.Menu.DataBindItem(MenuItem item) +56

System.Web.UI.WebControls.Menu.PerformDataBinding() +50

System.Web.UI.WebControls.HierarchicalDataBoundControl.PerformSelect() +85

System.Web.UI.WebControls.BaseDataBoundControl.DataBind() +73

System.Web.UI.WebControls.Menu.DataBind() +4

System.Web.UI.WebControls.BaseDataBoundControl.EnsureDataBound() +82

System.Web.UI.WebControls.Menu.EnsureDataBound() +40

System.Web.UI.WebControls.BaseDataBoundControl.OnPreRender(EventArgs e) +22

System.Web.UI.WebControls.Menu.OnPreRender(EventArgs e, Boolean registerScript) +15

System.Web.UI.WebControls.Menu.OnPreRender(EventArgs e) +25

Microsoft.SharePoint.WebControls.AspMenu.OnPreRender(EventArgs e) +142

System.Web.UI.Control.PreRenderRecursiveInternal() +80

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Control.PreRenderRecursiveInternal() +171

System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +842

Wednesday, March 10, 2010

What every SharePoint administrator needs to know about Alternate Access Mappings?

Taken from Blogs.msdn.com post at http://blogs.msdn.com/sharepoint/archive/2007/03/06/what-every-sharepoint-administrator-needs-to-know-about-alternate-access-mappings-part-1.aspx

--
Hi folks, this is Troy Starr again from the Windows SharePoint Services Test team. Today I'd like to talk about one of the most powerful, but often one of the least understood, features in Windows SharePoint Services 3.0 and Microsoft Office SharePoint Server 2007. That feature is called Alternate Access Mappings. Around here, we just call it "AAM" for short.

At the most basic level, AAM tells SharePoint how to map web requests (for example, browsing to the homepage of a SharePoint site) to the correct web application and site so that SharePoint can serve the correct content back to you. It then tells SharePoint what URL the users should be taken to as they interact with SharePoint.

Seems simple enough, doesn’t it? Those of you who are familiar with developing web applications in Internet Information Services may be wondering right now why we need such a feature since IIS can tell you what the URL of an incoming web request is. The major reason we need this is that there are common Internet deployment scenarios where the URL of a web request received by IIS is not the same URL that the end user entered. These are most common in reverse proxy publishing and load balancing scenarios.

How is this possible? Let's consider a reverse proxy scenario. A reverse proxy is a device that sits between end users and your web server. All requests to your web server are first received by the reverse proxy device, and if those requests pass the proxy's security filtering, the proxy will forward the requests on to your web server. Reverse proxies can perform advanced functionality such as receiving a web request over the Internet via SSL (HTTPS), but forward the request to the your server via HTTP. This is referred to as off-box SSL termination. They can also forward the request to a different port number than it was originally received on and can even change the HTTP Host header field. If SharePoint were base its links off of the URL of the request it received, the links that end users click on could be the incorrect "internal" URL rather than the correct "public" URL.

SharePoint is compatible with a variety of reverse proxy servers, but for this example we'll take a look at a publishing rule from Microsoft's reverse proxy software - Internet Security and Acceleration Server 2006. ISA Server 2006 includes a SharePoint publishing wizard that walks you through creating a publishing rule for SharePoint. Once the rule is created, you can modify it at any time. (The following images show a slightly modified publishing rule where the "Forward the original host header" option is turned off to help demonstrate the flexibility of AAM. If we left the "Forward the original host header" option turned on, the public hostname would also serve as the internal hostname when configuring AAM.) The first two dialogs show the "listener" and "public name" properties of the rule, which define what URL users will use to access your SharePoint site. Remember that this URL is really the URL of your reverse proxy server, which will forward the request to your SharePoint server.






The end user's URL is comprised of the public protocol, the public hostname, and the public port number.

Public Protocol
+
"://"
+
Public Hostname
+
":"
+
Public Port Number
=
Public URL

HTTPS
www.contoso.com
443
https://www.contoso.com


The next two dialogs show the "to" and "bridging" properties of the rule, which define what URL the reverse proxy server will use to forward the request to your SharePoint server.





The SharePoint server's URL is comprised of the internal protocol, the internal hostname, and the internal port number.

Internal Protocol
+
"://"
+
Internal Hostname
+
":"
+
Internal Port Number
=
Internal URL

HTTP
sharepoint.dmz.contoso.com
80
http://sharepoint.dmz.contoso.com




Great - we've properly set up this reverse proxy server to receive web requests from end users at https://www.contoso.com and forward them to your SharePoint server at http://sharepoint.dmz.contoso.com. We're halfway there! The next step is to configure your SharePoint web application and AAM to match the publishing rule above. To do this, we'll extend an existing web application to an additional IIS Web site just for your reverse proxy's publishing rule. Note that you're also able to create a new web application from scratch for this publishing rule - the fields you'll need to fill out are the same in either case.

Browse to your WSS 3.0 Central Administration site and click on the Application Management tab. Next, click "Create or extend Web application" and then click "Extend an existing Web application." Select the web application that you wish to use, and then fill out the port, host header, and SSL fields based on the internal URL properties that we defined above. In the URL field, enter the public URL that we defined above. Finally, you'll want to select an AAM Zone to assign this extension of your web application to. There are a maximum of 5 zones available in each web application. We'll use the Internet zone in this example, but you're free to use any available zone. All of the zones provide the same functionality, although the Default zone will always be used for certain features such as sending administrative e-mail messages to site collection owners. When you're finished, click OK to create the IIS Web site.



Next, you'll want to verify that your public URL was created correctly in AAM and then add your internal URL. Unless your internal URL is the same as your public URL, this is an extra step that you must perform manually. To do this, browse to your WSS 3.0 Central Administration site and click on the Operations tab. Next, click "Alternate access mappings." Click the Alternate Access Mappings selector drop-down and select the web application that you're publishing through your reverse proxy server. You should now see the AAM URLs assigned to your web application.




As you can see, the public URL from the reverse proxy publishing rule has been assigned to your web application's Internet zone. The final touch is to add the internal URL from the reverse proxy publishing rule to your web application's Internet zone. To do this, click "Add Internal URLs" in the AAM toolbar, type in the internal URL, and select the same zone that you used for the public URL. In this case, that was the Internet zone. When you're finished, click Save. You should now see the additional URL is assigned to your web application, in the same zone as the public URL of your reverse proxy publishing rule.




All done! Now, when a user browses to https://www.contoso.com, the web request will be received by the proxy server and forwarded to http://sharepoint.dmz.contoso.com. SharePoint will then receive the web request, see that the URL of the request is http://sharepoint.dmz.contoso.com, find that this URL is assigned to the Contoso web application, and return the content from that web application. In addition, because the http://sharepoint.dmz.contoso.com URL is assigned to the Internet zone, SharePoint will also generate links on the pages using the public URL for that zone - https://www.contoso.com. This ensures that end users are taken to the proper URL when clicking on links on the page.

Load balancers work similarly, especially if they overwrite the end user's original URL with the URL of the individual web server that the request is being load balanced to. To address this, just add each individual web server's URL to AAM as an internal URL and associate them all to the same zone as end user's public URL.

I hope that this introduction to Alternate Access Mappings was helpful to you. Please feel free to post comments to this blog entry with any questions you may have about AAM. I will be posting another blog entry soon covering common AAM mistakes and how to avoid them.



Troy Starr

How to backup VMWare Virtual Machines?


As you may already know - backing up virtual machines isn't a walk in the park. Actually, it is very much similar to usual backup process of any physical machine. However, VMWare has a nice tool called "VMware Consolidated Backup" which can do even online backups of virtual machines, without disrupting users and applications. Of course this tool isn't free. So, we are back at the begining of this story - how to backup virtual machines.

There are few ways to do manual backup of virtual machines which can then bi scripted into a semi-automatic backup or automatic backup if you whish. The down side of this approach is that you have downtime of virtual machine when doing backup, but in most cases I find this acceptable.

We are using VMWare server 2.0 platform for our virtual machines. There are few requirements in order to do this. First, you need to familiarize yourself with vmrun command line tool. Vmrun is used for managing virtual machines from command line. This and other Vmware tools are by default located in c:\Program Files\VMWare\VMware Server\.

This is what I use for "soft" stopping virtual machine SPWFE1.domain.local located at E_DRIVE datastore of VMWare Server named VM.domain.local. I am using local administrator user context for running this command i.e. user:admin with password:wordxxxx

From command prompt run: "vmrun -T server -h https://vm.domain.local:8333/sdk -u admin -p wordxxxx stop "[E_DRIVE] spwfe1\spwfe1.domain.local.vmx" soft"

This command, shutsdown virtual machine spwfe1.domain.local after ending all processes.

Now, we need to go to next step, which is copying the virtual machine to another location i.e. making backup of the virtual machine. In order to do that - I use xcopy command to copy folder content to another location.

After copying is done, we can start the virtual machine by running command "vmrun -T server -h https://vm.domain.local:8333/sdk -u admin -p wordxxxx start "[E_DRIVE] spwfe1\spwfe1.domain.local.vmx"

The final thing, in order to automate this, you should organize stop, copy and startup scripts into different scripts and use either windows AT command or Scheduled Tasks to schedule starting of these scripts.

Sharepoint system architecture


When thinking about sharepoint architecture - you got to first think about the platform architecture and system design. Nowdays, virtualization technology is something we all need to think about when planning system design of any kind. That goes also for sharepoint architecture.

Sharepoint, as we all know is Microsoft's content management system and collaboration tool for groupwares. As such, it is used very often as company's Intranet or Internet presence platform. Since Sharepoint allows you to distribute it's components through many hardware boxes in order to load balance requests - it can be used in so called multi-layerd architecture also known as multi-tier architecture. In multi-tier architecture - Sharepoint can be installed in load-balancing and fail-over environment to provide better performance and high availability.

So, ideal Sharepoint architecture is comprised of multiple hardware boxes serving in multi-tier architecture. Virtualization technology fits well in this multi-box multi-tier architecture. There is only one rule you need to know about - virtualization of database servers is not something you should do or use in production environment. The main reason is because DBs often do a lot I/O disk activity which doesn't perform so well in virtual environment. On the other hand - it is ok to have virtualized DB server in testing environment.

Sharepoint backup error: Object xyz_DB failed in event OnBackup

I started having problems with Sharepoint backup some time ago. The problems were that I would get an error message "Object SharedServices1_Search_DB failed in event OnBackup" and similar started showing in Central Administration backup when I moved my backup location from Sharepoint server to UNC network location. I tried many things but - I even tried to login with Sharepoint service account and do the backup but it didn't work - even though this same account "spservice" had "Full control" on this network share folder. So, what eventually did help me was to (for testing purposes only) change NTFS permissions on network share where my backup was done. I gave Everyone and Anonymous group "Full control" permissions on my \\fileserver\StsadmBackup folder. After I did that - backup was done succesfully! :)

Anyway, at least now I know that everything is OK with my Sharepoint frontend and DB servers. It was permissions that were the issue. I will probably move backup folder back to frontend machine or I will leave UNC path on network share but really need to get rid of anonymous and everyone groups.

Regards,
Damir.