Biztalk 2006 updates & SP

BizTalk Server 2006 R2 

Cumulative Update

BizTalk Server 2006 R2 Service Pack 1 CU4

Version: 3.6.2237.12 
Release date: March 30, 2012

For more information, click the following article number to view the article in the Microsoft Knowledge Base: 

2682056 Cumulative update package 4 for BizTalk Server 2006 R2 Service Pack 1

BizTalk Server 2006 R2 Service Pack 1 CU3

Version: 3.6.2224.12
Release date: August 30, 2010

For more information, click the following article number to view the article in the Microsoft Knowledge Base: 

2286501 Cumulative update package 3 for BizTalk Server 2006 R2 Service Pack 1

BizTalk Server 2006 R2 Service Pack 1 CU2

Version: 3.6.2217.12
Release date: June 24, 2010

For more information, click the following article number to view the article in the Microsoft Knowledge Base: 

2211420 Cumulative update package 2 for BizTalk Server 2006 R2 Service Pack 1

BizTalk Server 2006 R2 Service Pack 1 CU1

Version: 3.6.2210.12
Release date: April 12, 2010

For more information, click the following article number to view the article in the Microsoft Knowledge Base: 

982061 Cumulative update package 1 for BizTalk Server 2006 R2 Service Pack 1

Service Pack

BizTalk Server 2006 R2 SP1 

Version: 3.6.2149.10
Release date: January 27, 2010

For more information about the hotfixes that are included in Microsoft BizTalk Server 2006 R2 Service Pack 1, click the following article number to view the article in the Microsoft Knowledge Base:

974563 List of Microsoft BizTalk Server hotfixes that are included in BizTalk Server 2006 R2 Service Pack 1

BizTalk Environment Maintenance from a DBA perspective

from saravana's article:

This is one of the common challenges we see in many enterprises. SQL servers will always be owned by a dedicated database team. The bigger the organisation, the partition between the BizTalk support team and database team will increase. One of the biggest challenges with this setup is, from DBA's perspective they wanted to follow all the best practices, they normally follow as a standard for all the SQL Servers/Databases in the organisation. These may include things like standard backup procedures, indexing procedures, standard recovery model, their own SQL bookkeeping jobs etc.

This situation is a real red alert, BizTalk databases are special databases. A typical BizTalk installation at the minimum will have 4 databases BizTalkMsgBoxDb, BizTalkMgmtDb, BizTalkTrackingDb, SSODB. They are designed to work as a single group. There are certain dependencies between them. Example: Some of the SQL jobs are designed to move data from BizTalk message box to tracking DB, the Tracking subservice will also do similar thing to move data from message box. There are lot of chances for distributed transactions spanning across these databases.  So, the general rule of thumb is

"Do not treat your BizTalk databases as your standard database. It's just a black box and you are not allowed to make any changes (there are few exceptions)"

BizTalk server by default comes with all the weapons required to maintain/protect its databases. The product is designed in a way to self maintain. The main challenge here is SQL DBAs are not fully aware of the BizTalk capabilities.  In this article we will see some of the routine jobs SQL DBA's must be aware of when maintaining a BizTalk environment related databases.

1.    Know how to use Message Box Viewer/Terminator
2.    One time settings
3.    Make sure SQL Agent is running
4.    Procedures to rebuild indexes for BizTalk databases
5.    Monitor the growth of certain tables
6.    Monitor the size of databases
7.    Monitor transaction log sizes
8.    BizTalk 2010 Monitor BizTalk Server Job

Know how to use Message Box Viewer/Terminator 
Message box viewer and Terminator tool are the two tools the DBA's maintaining the BizTalk environment should be completely aware of. MBV is designed to identify all potential problems in the BizTalk environment that could be critical or need attention and present them in a nice report. Most of the BizTalk environment issues will come down to poorly maintained database(s); MBV is extremely good at picking them up. Full Q&A can be found here 
http://blogs.technet.com/b/jpierauc/archive/2008/07/22/msgboxviewer-mbv-q-a.aspx

In earlier days people executed MBV, identified the issues and manually corrected the problems. But with latest version of MBV (10.3 and above), MBV produce a cleanup script, which can then be used with another tool called Terminator to fix the database issues either automatically or manually.

http://blogs.msdn.com/b/biztalkcpr/archive/2011/02/10/using-biztalk-terminator-to-resolve-issues-identified-by-biztalk-msgboxviewer.aspx

One time settings 
Here are some of the steps you need to perform once the environment is setup.

1. Install the Latest BizTalk Server Service Pack and Cumulative Update

2. Disable the Auto Update Statistics and Auto Create Statistics options

You must disable the Auto Create Statistics and Auto Update Statistics options on theBizTalkMsgBoxDb database. To determine whether these settings are disabled, execute the following stored procedures in SQL Server:
exec sp_dboption 'BizTalkMsgBoxDB', 'auto create statistics'
exec sp_dboption 'BizTalkMsgBoxDB', 'auto update statistics'

You should set the CurrentSetting setting to off. If this setting is set to on, turn it off by executing the following stored procedures in SQL Server:

exec sp_dboption 'BizTalkMsgBoxDB', 'auto create statistics', 'off'
exec sp_dboption 'BizTalkMsgBoxDB', 'auto update statistics', 'off'

3. Set the Max Degree of Parallelism property correctly

On the computer that is running SQL Server and hosting the BizTalkMsgBoxDb database, set the Max Degree of Parallelism run_value and config_value properties to a value of 1. To determine the Max Degree of Parallelism setting, execute the following stored procedure against the Master database in SQL Server:

exec sp_configure 'max degree of parallelism'

If the run_value and config_value properties are not set to a value of 1, execute the following stored procedure in SQL Server to set them to 1:

exec sp_configure 'max degree of parallelism', '1'
reconfigure with override

4. Make sure BizTalkMsgBoxDb and BizTalkTrackingDb data and log files are in separate drives, also if possible make sure you don't put BizTalkMsgBox and BizTalkTrackingDb data files or log files in the same drive.

Rebuilding indexes 
The only supported method to rebuild an index in the BizTalkMsgBoxDb database is to run thebts_RebuildIndexes stored procedure. On BizTalk Server 2006 and later versions, you can run thedtasp_RebuildIndexes stored procedure to rebuild indexes in the BizTalkDTADb database.

If lots of data is expected to build up in the BizTalkMsgBox database, you can periodically rebuild indexes during scheduled downtime. The same guidelines apply to the tracking database.

You can use the DBCC DBREINDEX SQL command to rebuild an index in the other BizTalk Server databases. For an example of how to use the DBCC DBREINDEX SQL command, right-click thebts_RebuildIndexes stored procedure, and then click Properties.
http://support.microsoft.com/default.aspx?scid=kb;EN-US;917845 

Monitor the growth of certain tables Message Box Database (BizTalkMsgBoxDb) 
Spool:
If the Spool tables have many records, many messages are currently active, dehydrated, or suspended. Depending on the size, the number of parts, and the fragmentation settings in these tables, a single message may spawn all these tables. Each message has exactly one row in the Spool table and at least one row in the Parts table. Create a benchmark for your BAU activities and make sure the spool table is not growing indefinitely. During you quite period when there are no BAU activities if you notice a large number of rows in this table, make sure unwanted suspended instance are cleared periodically. You can also use Message Box Viewer regularly to check if there are any unusual numbers of rows in the table and can use Terminator tool to safely remove them.

TrackingData_0_x and TrackingData_1_x TrackingData_0_x: These four tables store the Business Activity Monitoring (BAM) tracked events in the BizTalkMsgBoxDb database for TDDS to move the events to the BAMPrimaryImport database.
TrackingData_1_x: These four tables store the tracked events in the BizTalkMsgBoxDb database for TDDS to move the events to the BizTalkDTADb database.

You make sure there is at least one host configured with tracking enabled. This will run TDDS subservice responsible for moving the data from message box db to BizTalkDTADb and BAMPrimaryImport 

BizTalk Tracking Database (BizTalkDTADb) dta_DebugTrace table typically becomes large in environments where orchestration shapes start/end is enabled. If there is no business value and if you got lots of orchestration in your solution, it's better to clear the check box for ?Shape start and end? option in the orchestration(s) properties.

dta_MessageInOutEvents table typically becomes large in environments where ?Message send and receive? is enabled for orchestrations and/or pipelines. If these tracking events are not required, clear the check box for this option in the orchestration and/or pipeline properties.

If the dta_DebugTrace table and/or the dta_messageInOutEvents table in the BizTalkDTADb database are too large, you can truncate the tables manually after you stop the tracking host. The BizTalk Terminator tool also provides this functionality.

dta_ServiceInstanceExceptions table typically becomes large in an environment that regularly has suspended instances. dta_serviceinstances table grows every time the user terminates an instance. DTA Purge and Archive SQL Server jobs should take care of clearing this table, so make sure the jobs are running correctly.

BAMPrimaryImport
TDDS_FailedTrackingData The TDDS_FailedTrackingData table gets populated whenever there is a tracking failure or even in cases where you haven't deployed your BAM activities, but your solution is trying to insert BAM data. In earlier version of BizTalk (2006 and 2006 R2) the DTA Purge and Archive SQL Server jobs didn't clear this table periodically which resulted in unlimited growth of this table. Make sure you got the hot fix explained here http://support.microsoft.com/kb/977289 

Monitor the size of databases The size of the databases will vary from organisation to organisation depending on the volume of data processing. In any organisation having a bloated/large database will adversely effects the performance of your BizTalk environment. As a general rule of thumb, you can use the following numbers for guidance

BizTalkMsgboxDb: Not more than 5GB. However big your deployment is, your message box database should NOT go beyond this limit. In theory Message Box database only should hold transit data (aka inflight messages).  On a healthy environment all the processed message will either be moved to BizTalk tracking database or BAM database. Having lot of suspended instances will also bloat the size of the message box database, so periodically clear them out or better design your solution in a way you are not going to have too many suspended instances (unless otherwise there is a genuine unhanded exception scenario).

BizTalkDTADb, BAMPrimaryImportDB: Around 10 GB max.  This is again a finger in the air estimate. It's better to keep the tracking data and BAM data within this limit. You can control this by setting the appropriate values for your DTA purging/archiving job and making sure you are moving your BAM data from BAMPrimaryImport table to BAMArchieve database. Also make sure yourBAMArchive database is not bloated over a period of time. Do a hard purge after a set period like 3 or 6 months based on your business requirement.

The other databases like BizTalkMgmtDb, SSO, RulesEngine etc all store configuration data and they should be any bigger than 2GB. 

Monitor transaction log sizes The size of the BizTalk databases transaction log is controlled by the BizTalk backup job. By default the recovery model for the BizTalk databases is set to Full. If the transaction log is not backed up or truncated on a regular basis, the log files or files can fill up. So, please make sure BizTalk backup job is configured and running successfully. This job automatically backup the BizTalk databases including transaction log, and thus ensures the transaction log does not grow to an unmanageable size. The backup job should also be performed multiple times during the day as it's the job that lets you recover your messages. The bigger the log file the more messages that could be lost.

It is not recommended to change the recovery model settings of the BizTalk databases. Changing this setting will put the BizTalk environment into a state where it may not be fully recoverable in the event of failure.

Not but the not the least the Backup BizTalk Server job is the only supported method to back up the BizTalk Server databases.

BizTalk 2010 Monitor BizTalk Server Job If you are using BizTalk Server 2010, you can run the Monitor BizTalk Server SQL Agent job to identify any known issues in Management, Message Box, or DTA databases. The job is created when you configure a BizTalk group in BizTalk Server Administration console or upgrade BizTalk from the previous version.

The Monitor BizTalk Server job scans for the following issues in Management, Message Box, and DTA databases:

• Messages without any references
• Messages without reference counts
• Messages with reference count less than 0
• Message references without spool rows
• Message references without instances
• Instance state without instances
• Instance subscriptions without corresponding instances
• Orphaned DTA service instances
• Orphaned DTA service instance exceptions
• TDDS is not running on any host instance when global tracking option is enabled.

The Monitor BizTalk Server job is configured and automated to run once in a week. Since the job is computationally intensive, it's recommended you to schedule it during downtime/low traffic.

The job fails if it encounters any issues; error string contains the number of issues found. Else, it runs successfully. You can see the details in the job history. If you run the job with Administrator privileges, error string will be logged to Event Viewer also (along with the job history).

Ad-hoc Full backup It may be required once in a while to force a full database backup. TheBizTalkMgmtDb.dbo.sp_ForceFullBackup stored procedure can be used to force a full backup of the data and log files. Execute this stored procedure, and then execute the Backup BizTalk Server SQL Agent job.

Cleanup all the data in test environment DO NOT do this in PRODUCTION
When you are managing test environment (especially performance testing) it may be required to clean up BizTalkMsgBoxDb and BizTalkDTADb. When I say clean up, completely wipe out all the data, hence the bold warning.

BizTalkMsgboxDb
1.    Copy the Msgbox_cleanup_logic.sql script from Drive:\Program Files\Microsoft BizTalk 200x\schemato the SQL Server.
2.    Execute this SQL script against the BizTalkMsgBoxDb database to update the bts_CleanupMsgboxstored procedure.
3.    Stop all BizTalk hosts, services, and custom isolated adapters. If you use HTTP or the SOAP adapter, restart the IIS services.
4.    Execute the bts_CleanupMsgbox stored procedure on all the BizTalkMsgBoxDb databases.
5.    Restart all host instances and BizTalk Server services.

BizTalkDTADb 
1.    Back up all BizTalk databases.
2.    Execute the dtasp_CleanHMData stored procedure. Only use this option if the BizTalkDTADbdatabase contains many incomplete instances that must be removed.

To do this, follow these steps:
a.    Stop all BizTalk hosts, services, and custom isolated adapters. If you use HTTP or the SOAP adapter, restart the IIS services.
b.    Execute the dtasp_CleanHMData stored procedure on the BizTalkDTADb database.
c.    Restart all hosts and BizTalk Server services.

FTP Receive Adapter fails to publish message if the receive pipeline processing time exceeds the server timeout value

Problem

Pipeline processing is performed on all messages that are received by the FTP Adapter. If the time taken to complete pipeline processing for messages received by the FTP connection times out then the message will not be published to the Messagebox and errors similar to the following may be generated in the Application log of the BizTalk server:

other

Event Type:Error
Event Source:BizTalk Server 2006
Event Category:BizTalk Server 2006 
Event ID:5719
Date:8/30/2006
Time:12:08:55 PM
User:N/A
Computer:BIZTALKSERVER
Description:
There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.PassThruReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "Pipeline " Receive Port: "ReceivePort1" URI: "ftp://FTPSERVER:21/*.txt" Reason: Unable to receive the file "file.txt   " from the FTP server. 

and

other

Event Type:Warning
Event Source:BizTalk Server 2006
Event Category:BizTalk Server 2006 
Event ID:5740
Date:8/30/2006
Time:12:08:56 PM
User:N/A
Computer:BIZTALKSERVER
Description:
The adapter "FTP" raised an error message. Details "Unable to receive the file "file.txt   " from the FTP server. ".

Cause

The FTP Adapter attempts to delete the original copy of the message on the source FTP Server before it publishes the message to the Messagebox. If the connection has timed out then the FTP Adapter cannot delete the copy of the message on the source FTP Server.

Resolution

Use one of the following methods to mitigate this problem:

1. Increase the connection timeout value. Set the Idle timeout value on the FTP server to be at least the amount of the time it takes to process the file.
   
2. Use the Temporary Folder feature on the receive location. In this case, the FTP adapter copies the file to the temporary folder (typically local disk). The time to copy the file to local disk is less than the time it takes run the message through pipeline and persist it to the message box which effectively reduces the idle time.

Data may be duplicated or lost when you receive data in BizTalk Server 2006 by using the FTP adapter

Problem

Data is duplicated or lost when you receive data in BizTalk Server 2006 by using the FTP Adapter.

Cause

The BizTalk Server 2006 FTP adapter uses the FTP client protocol to poll the designated FTP server and retrieves data from the server "as is." The FTP adapter does not validate any data that it retrieves. The FTP adapter sends the retrieved document to the BizTalk Messaging Engine for processing and then it deletes the original document from the FTP server. If the FTP adapter retrieves a document from the FTP server that is still being written to by the host application, the retrieved document will be incomplete. If the FTP adapter retrieves an incomplete copy of the original document, data duplication or data loss may occur in the following scenarios:

• If the original document is still being written to the FTP server by the host application, the FTP adapter cannot delete the document and will retrieve another copy of the document at the next polling interval that is configured for the receive location. This behavior causes document duplication to occur.
  
• If the host application has finished writing the document to the FTP server, the document will be deleted. This behavior will cause data loss to occur.
  

Resolution

To work around this behavior, use one of the following methods:

• Configure the host application to write to a temporary folder on the same hard disk as the public FTP folder and to periodically move the contents of the temporary folder to the FTP folder. The temporary folder should be on the same hard disk as the public FTP folder to make sure that the move operation is atomic. An atomic operation is an operation that is functionally indivisible. If you write data to the public FTP folder by using the BizTalk Server 2006 FTP adapter, you can do this by specifying a Temporary Folder property in the FTP Transport Properties dialog box when you configure a send port. If you specify a Temporary Folder property, make sure that this folder is on the same physical disk as the public FTP folder.
  
• Configure the FTP receive location to operate within a service window when the host application is not writing data to the FTP server. You can specify the service window when you configure the receive location properties.

FIX: A file is not removed from an FTP location when you use BizTalk Server 2006 R2

Consider the following scenario in Microsoft BizTalk Server 2006 R2 or BizTalk Server 2009:

• You retrieve a file from a subfolder at an FTP location by using an FTP receive location.
• While BizTalk Server 2006 R2 is processing this file, the FTP connection times out.
• After some time, BizTalk Server 2006 R2 opens a new FTP session, and then issues a DELE command.

In this scenario, BizTalk Server 2006 R2 cannot delete the received file from the subfolder at the FTP location. Additionally, the new FTP session is closed, and you can see log entries in the BizTalk Server 2006 R2 FTP log that resemble the following:


< 220 Microsoft FTP Service
> USER anonymous
< 331 Anonymous access allowed, send identity (e-mail name) as password.
> PASS xxxx
< 230 Anonymous user logged in.
> PWD
< 257 "/" is current directory.
> DELE file_name
< 550 file_name: The system cannot find the file specified.
> QUIT
< 221


When the BizTalk Server 2006 R2 or BizTalk Server 2009 next polls the FTP location, BizTalk Server 2006 R2 or BizTalk Server 2009 opens the FTP location and downloads the same file again.


CAUSE:
This problem occurs because BizTalk Server 2006 R2 or BizTalk Server 2009 does not issue a CWD command before it issues the DELE command. Therefore, instead of trying to delete the file from the configured subfolder, BizTalk Server 2006 R2 or BizTalk Server 2009 tries to delete the file from the FTP root directory.


HOTFIX:
The hotfix that resolves this problem is included in cumulative update package 1 for BizTalk Server 2006 R2 SP1. 

How to Export Bindings for a BizTalk Application (without Admin console)

1. Open a command prompt as follows: Click Start, click Run, type cmd, and then click OK.
   
2. Type the following command, substituting the appropriate values, as described in the following table:
   
   
   BTSTask ExportBindings /Destination:value [/ApplicationName:value] [/GlobalParties] [/Server:value] [/Database:value]
   
   
   Example:
   
   
   BTSTask ExportBindings /Destination:"C:\Binding Files\MyBindings.xml" /ApplicationName:MyApplication /Server:MyDatabaseServer /Database:BizTalkMgmtDb

   
   
   
   

   Parameter Value:

   
   

   /Destination
   Full path of the binding file to create, including the file name. If a binding file having the same path already exists, it is overwritten. If there are spaces in the path, you must enclose it in double quotation marks (").
   /ApplicationName
   Name of the application from which to export bindings. The application must exist. To verify the application name, you can use the ListApps command, as described in ListApps Command. If this parameter is not specified, the default BizTalk application is used. If there are spaces in the name, you must enclose it in double quotation marks (").
   /GlobalParties
   When specified, exports global party information for the group.
   /Server
   Name of the SQL Server instance hosting the BizTalk Management database, in the form ServerName\InstanceName,Port.
   Instance name is only required when the instance name is different than the server name. Port is only required when SQL Server uses a port number other than the default (1433).
   Examples:
   Server=MyServer
   Server=MyServer\MySQLServer,1533
   If not provided, the name of the SQL Server instance running on the local computer is used.
   /Database
   Name of the BizTalk Management database. If not specified, the BizTalk Management database running in the local instance of SQL Server is used.
   

   
   

Biztalk 2006: Oracle adapter namespace problem

I was deploying a Biztalk project from the development enviroment to the test enviroment when I encountered an error with Oracle adapter. That's because I have a different database schema name in both enviroments and database schema name is hard-coded to the schema.


My solution:


Open a file of the solution( ex. xxxx.xsd) with editor xml (in visual studio) , and make a find of NAMEOFSCHEMA and make a REPLACE in entire solution with new VALUE of NAMEOFSCHEMA.


Redeploy , change the port that uses the WCF adapter oracle, set the new NAMEOFSCHEMA  in soap Action value...


i've tried other solutions such as delete the namespace and activate EnableBiztalkCompabilityMode=true anduseSchemaInNamespace=false, but without success...




bye

MSMQ and Biztalk: larger message of 4 mb

in working...

Using system.byte[] in map biztalk 2006

Hi
when you using a byte[] into map, you make or in script functoid or in dll .net this:
public string GETXXX(string fileurl)
byte[] content = null;
try
    {
     System.IO.FileInfo file = new System.IO.FileInfo(fileurl);
      content = System.IO.File.ReadAllBytes(file.FullName);
       }
       catch (Exception ex)
      {
       content = null; //optional , your logic ....
       System.Diagnostics.Debug.WriteLine(ex.Message); //optional
       }
return System.Convert.ToBase64String(content); //this is the code right!! into the map the destination field is a BINARY but you pass a string converted!

bye