Upload - Files and forms with large/huge size. Article

Brief overview of work with very large files. ASPForm configuration and IIS setting for huge uploads or urlencoded forms with big fields.

Main idea for big uploads

Upload components (including ASPNet HTMLInputFile) usually use memory to store temporary upload data. It is a problem, if you are uploading very large files.

Huge-ASP upload works with unique hi-performance technology, which lets you read upload source data in small data blocks (Request.BinaryRead) and store the data on temporary disk folder - this is automatic Huge-ASP upload work, of course. You can specify size of a source data block and maximum amount of memory to store data (to store small uploads in memory).

For example, HTMLInputFile from ASPNet takes around 5MB of memory to upload 1MB file. You will need at least 500 MB of a free memory if you have 10 clients uploading one 10MB file with ASPNet HTMLInputFile upload control. Huge-ASP upload takes less than 1MB (one megabyte) of memory to do the same action (and 100MB disk size to store temporary data, of course).

Important points for huge uploads with Huge-ASP upload

1. Server.ScriptTimeout and ScriptUtils.ASPForm.ReadTimeout property.
Big uploads take much time to complete. Next table contains approximate time values to upload files:

File size	Line speed [kb/s]	Upload time	Upload time [s]
1MB	33	00:04:33	273
10MB	33	00:45:27	2727
100MB	33	07:34:33	27273
1MB	64	00:02:05	125
10MB	64	00:20:50	1250
100MB	64	03:28:20	12500
1 000MB	64	34:43:20	125000
1MB	256	00:00:35	35
10MB	256	00:05:52	352
100MB	256	00:58:36	3516
1 000MB	256	09:45:56	35156
1MB	8000	00:00:01	1
10MB	8000	00:00:11	11
100MB	8000	00:01:52	113
1 000MB	8000	00:18:45	1125
2 000MB	80000	00:03:45	225
4 000MB	80000	00:07:30	450

Huge-ASP upload has a great performance - it lets you really accept 2GB files in 200second on 100Mb/s Ethernet line. But you have to set ScriptTimeout value to have enough time to transfer data and some time more to process uploaded files on server-side. If you want to accept up to 1GB uploads from min. 256k leased lines, set at least:

      Server.ScriptTimeout = 40000

There is no problem to set a big value for ScriptTimeout - Huge-ASP upload checks client connection during upload and server-side upload process is stopped when client interrupts upload.
You can also change this value by some dynamic algorithm with parameter of source form size. For example, on 10Mb/s Ethernet line (real 8000kb/s, 1MB/s):

Dim ScriptTimeout, TotalBytes
TotalBytes = Request.TotalBytes
'TotalBytes/1000000 - time to transfer data
'TotalBytes/5000000 - time to save data on server-side

ScriptTimeout = 100 + TotalBytes/1000000 + _
 TotalBytes/5000000
Server.ScriptTimeout = ScriptTimeout

2. ScriptUtils.ASPForm.SizeLimit Size limit
      Second important value is a ScriptUtils.ASPForm.SizeLimit property. Huge-ASP upload has no limit set by default. It means you can upload and accept any files up to 2GB of size (2^31-1B; Huge-ASP upload can accept up to 4GB, but you will probably not have a client which let's you upload files bigger than 2GB ...).
      Internet Explorer (from v 5.0) and Netscape Navigator let's you upload files with size up to 2GB also.
      Be careful to have this value without limit - upload scripts are possible point for DOS attacks. If you have public upload script, set lower value rather. You can also create some dynamic algorithm to limit upload size based on logged on users, local IP addresses, etc.

Form.SizeLimit = GetLimitFromDB(LOGON_USER)

Please see ScriptUtils.ASPForm.Upload - Monitor and handle upload state/result article to handle "upload size exceeds the limit" state.

3. ScriptUtils.ASPForm.TempPath Temporary path
Temporary upload data are stored in predefined folder. Please, set a folder with adequate free size. For example, if you have up to 10 clients uploading up to 1GB file, you need to have 10GB temporary folder to store temporary form data (and second 10GB free space on destination disk/store).

4. IIS 6 and AspMaxRequestEntityAllowed
The AspMaxRequestEntityAllowed property specifies the maximum number of bytes allowed in the entity body of an ASP request. The default value for the AspMaxRequestEntityAllowed property is 204800 bytes (200kB). You can get the current value using adsutil.vbs:

cscript adsutil.vbs get w3svc/ASPMaxRequestEntityAllowed

Please increase this value if you are uploading large files in IIS6
You can increase this value using set command (AppID 1, limit 200MB):

cscript adsutil.vbs set w3svc/1/ASPMaxRequestEntityAllowed 204800000

5. IIS 7 and maxAllowedContentLength
maxAllowedContentLength value specifies the maximum length of content in a request, in bytes. The default value is 30 000 000 (30MB). Please increase this value using web.config for IIS7.

6. URLScan and other security utilities URLScan has its own limit for POST data. The value name is MaxAllowedContentLength and it is located in WINDOWS\system32\inetsrv\urlscan\urlscan.ini, default value is 30000000 (30MB). You shlould increase the value to process large uploads if you have URLScan on your site. See also MaxAllowedContentLength urlscan.

7. Progress bar
Use progress bar every time you want to accept big uploads. Client browsers have no progress indicator by default. And clients-people know probably nothing about this problem. You will have many people, which will try to upload same files again and again (they will think nothing changed after "upload button" click) - without progress bar.
Huge-ASP progress bar is open immediately after Form submit button and client can see that something happened - and can see progress of upload.

maxRequestLength - Maximum request length exceeded - IIS6

Symptoms

When you try to use the ASPForm object to upload a large file with ASP and IIS6, the file may not be uploaded. This problem occurs because the default value for the maxRequestLength parameter in the <httpRuntime> section of the Machine.config file is 4096 (4 megabytes). As a result, files that are larger than this value are not uploaded by default.

Resolution

To resolve this problem, use one of the following methods:

In the Machine.config file, change the maxRequestLength attribute of the <httpRuntime> configuration section to a larger value. This change affects the whole computer.

In the Web.config file, override the value of maxRequestLength for the application. For example, the following entry in Web.config allows files that are less than or equal to 8 megabytes (MB) to be uploaded:

<httpRuntime maxRequestLength="8192" />

Web resources

http://support.microsoft.com/default.aspx?scid=kb;EN-US;295626
http://www.google.com/search?q=maxRequestLength

Other references

ScriptUtils.ASPForm.MaxMemoryStorage
ScriptUtils.ASPForm.ChunkReadSize
ScriptUtils.ASPForm.Upload - Monitor and handle upload state/result

IIS Limit Check

IIS Limit Check is a simple application, which let's you check main IIS parameters important for upload files.

Upload - Files and forms with large/huge size. Article