FTP Receive Adapter fails to publish message if the receive pipeline processing time exceeds the server timeout value

Problem

Pipeline processing is performed on all messages that are received by the FTP Adapter. If the time taken to complete pipeline processing for messages received by the FTP connection times out then the message will not be published to the Messagebox and errors similar to the following may be generated in the Application log of the BizTalk server:

other

Event Type:Error
Event Source:BizTalk Server 2006
Event Category:BizTalk Server 2006 
Event ID:5719
Date:8/30/2006
Time:12:08:55 PM
User:N/A
Computer:BIZTALKSERVER
Description:
There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.PassThruReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "Pipeline " Receive Port: "ReceivePort1" URI: "ftp://FTPSERVER:21/*.txt" Reason: Unable to receive the file "file.txt   " from the FTP server. 

and

other

Event Type:Warning
Event Source:BizTalk Server 2006
Event Category:BizTalk Server 2006 
Event ID:5740
Date:8/30/2006
Time:12:08:56 PM
User:N/A
Computer:BIZTALKSERVER
Description:
The adapter "FTP" raised an error message. Details "Unable to receive the file "file.txt   " from the FTP server. ".

Cause

The FTP Adapter attempts to delete the original copy of the message on the source FTP Server before it publishes the message to the Messagebox. If the connection has timed out then the FTP Adapter cannot delete the copy of the message on the source FTP Server.

Resolution

Use one of the following methods to mitigate this problem:

1. Increase the connection timeout value. Set the Idle timeout value on the FTP server to be at least the amount of the time it takes to process the file.
   
2. Use the Temporary Folder feature on the receive location. In this case, the FTP adapter copies the file to the temporary folder (typically local disk). The time to copy the file to local disk is less than the time it takes run the message through pipeline and persist it to the message box which effectively reduces the idle time.

Data may be duplicated or lost when you receive data in BizTalk Server 2006 by using the FTP adapter

Problem

Data is duplicated or lost when you receive data in BizTalk Server 2006 by using the FTP Adapter.

Cause

The BizTalk Server 2006 FTP adapter uses the FTP client protocol to poll the designated FTP server and retrieves data from the server "as is." The FTP adapter does not validate any data that it retrieves. The FTP adapter sends the retrieved document to the BizTalk Messaging Engine for processing and then it deletes the original document from the FTP server. If the FTP adapter retrieves a document from the FTP server that is still being written to by the host application, the retrieved document will be incomplete. If the FTP adapter retrieves an incomplete copy of the original document, data duplication or data loss may occur in the following scenarios:

• If the original document is still being written to the FTP server by the host application, the FTP adapter cannot delete the document and will retrieve another copy of the document at the next polling interval that is configured for the receive location. This behavior causes document duplication to occur.
  
• If the host application has finished writing the document to the FTP server, the document will be deleted. This behavior will cause data loss to occur.
  

Resolution

To work around this behavior, use one of the following methods:

• Configure the host application to write to a temporary folder on the same hard disk as the public FTP folder and to periodically move the contents of the temporary folder to the FTP folder. The temporary folder should be on the same hard disk as the public FTP folder to make sure that the move operation is atomic. An atomic operation is an operation that is functionally indivisible. If you write data to the public FTP folder by using the BizTalk Server 2006 FTP adapter, you can do this by specifying a Temporary Folder property in the FTP Transport Properties dialog box when you configure a send port. If you specify a Temporary Folder property, make sure that this folder is on the same physical disk as the public FTP folder.
  
• Configure the FTP receive location to operate within a service window when the host application is not writing data to the FTP server. You can specify the service window when you configure the receive location properties.

FIX: A file is not removed from an FTP location when you use BizTalk Server 2006 R2

Consider the following scenario in Microsoft BizTalk Server 2006 R2 or BizTalk Server 2009:

• You retrieve a file from a subfolder at an FTP location by using an FTP receive location.
• While BizTalk Server 2006 R2 is processing this file, the FTP connection times out.
• After some time, BizTalk Server 2006 R2 opens a new FTP session, and then issues a DELE command.

In this scenario, BizTalk Server 2006 R2 cannot delete the received file from the subfolder at the FTP location. Additionally, the new FTP session is closed, and you can see log entries in the BizTalk Server 2006 R2 FTP log that resemble the following:


< 220 Microsoft FTP Service
> USER anonymous
< 331 Anonymous access allowed, send identity (e-mail name) as password.
> PASS xxxx
< 230 Anonymous user logged in.
> PWD
< 257 "/" is current directory.
> DELE file_name
< 550 file_name: The system cannot find the file specified.
> QUIT
< 221


When the BizTalk Server 2006 R2 or BizTalk Server 2009 next polls the FTP location, BizTalk Server 2006 R2 or BizTalk Server 2009 opens the FTP location and downloads the same file again.


CAUSE:
This problem occurs because BizTalk Server 2006 R2 or BizTalk Server 2009 does not issue a CWD command before it issues the DELE command. Therefore, instead of trying to delete the file from the configured subfolder, BizTalk Server 2006 R2 or BizTalk Server 2009 tries to delete the file from the FTP root directory.


HOTFIX:
The hotfix that resolves this problem is included in cumulative update package 1 for BizTalk Server 2006 R2 SP1.