Integrating BizTalk Server 2010 and Microsoft Dynamics CRM : Communicating from BizTalk Server to Dynamics CRM (part 1) - Building the BizTalk components

3/29/2013 4:35:59 PM

The way that you send requests from BizTalk Server to Dynamics CRM 2011 has changed significantly in this release. In the previous versions of Dynamics CRM, a BizTalk "send" adapter was available for communicating with the platform. Dynamics CRM 2011 no longer ships with an adapter and developers are encouraged to use the WCF endpoints exposed by the product.

Dynamics CRM has both a WCF REST and SOAP endpoint. The REST endpoint can only be used within the CRM application itself. For instance, you can build what is called a web resource&;&; that is embedded in a Dynamics CRM page. This resource could be a Microsoft Silverlight or HTML page that looks up data from three different Dynamics CRM entities and aggregates them on the page. This web resource can communicate with the Dynamics CRM REST API, which is friendly to JavaScript clients. Unfortunately, you cannot use the REST endpoint from outside of the Dynamics CRM environment, but because BizTalk cannot communicate with REST services, this has little impact on the BizTalk integration story.

The Dynamics CRM SOAP API, unlike its ASMX web service predecessor, is static and operates with a generic Entity data structure&;. Instead of having a dynamic WSDL that exposes typed definitions for all of the standard and custom entities in the system, the Dynamics CRM 2011 SOAP API has a set of operations (for example, Create, Retrieve) that function with a single object type. The Entity object has a property identifying which concrete object it represents (for example, Account or Contract), and a name/value pair collection that represents the columns and values in the object it represents. For instance, an Entity may have a LogicalName&;&; set to "Account" and columns for "telephone1", "emailaddress", and "websiteurl."

In essence, this means that we have two choices when interacting with Dynamics CRM 2011 from BizTalk Server. Our first option is to directly consume and invoke the untyped SOAP API. Doing this involves creating maps from a canonical schema to the type-less Entity schema. In the case of doing a Retrieve operation, we may also have to map the type-less Entity message back to a structured message for more processing. Below, we will walk through an example of this.

The second option involves creating a typed proxy service for BizTalk Server to invoke. Dynamics CRM has a feature-rich Solution Development Kit (SDK) that allows us to create typed objects and send them to the Dynamics CRM SOAP endpoint. This proxy service will then expose a typed interface to BizTalk that operates as desired with a strongly typed schema. An upcoming exercise demonstrates this scenario.

Which choice is best? For simple solutions, it may be fine to interact directly with the Dynamics CRM 2011 SOAP API. If you are updating a couple fields on an entity, or retrieving a pair of data values, the messiness of the untyped schema is worth the straightforward solution. However, if you are making large scale changes to entities, or getting back an entire entity and publishing to the BizTalk bus for more subscribers to receive, then working strictly with a typed proxy service is the best route. However, we will look at both scenarios below, and you can make that choice for yourself.

1. Integrating Directly with the Dynamics CRM 2011 SOAP API

In the following series of steps, we will look at how to consume the native Dynamics CRM SOAP interface in BizTalk Server. We will first look at how to query Dynamics CRM to return an Entity. After that, we will see the steps for creating a new Entity in Dynamics CRM.

Querying Dynamics CRM from BizTalk Server

In this scenario, BizTalk Server will request details about a specific Dynamics CRM "contact" record and send the result of that inquiry to another system.

Building the BizTalk components

In this first set of steps, we define the canonical BizTalk schema, add the Dynamics CRM schemas, create a mapping between the formats, and build an orchestration that ties all the pieces together:

In Visual Studio 2010, create a new BizTalk project named Chapter3-DynamicsCRM.AccountInt. This project will hold all the BizTalk artifacts for this solution. Right-click the project, choose Properties, and set the strong name key, and the deployment application to Chapter3.
Right-click the BizTalk project and choose to add a new item. Select the (XML) schema item type and name the schema Customer_XML.xsd.
This schema reflects our organization's internal definition of a customer. The result of our query to Dynamics CRM should be mapped to this structure. Our customers are defined by an identifier, name, set of addresses, set of phone numbers, and an email address. The following image reflects the structure of the schema:
Add a reference to the Dynamics CRM SOAP endpoint in order to generate the artifacts that we need to consume the service. The WCF SOAP endpoint for a Dynamics CRM 2011 instance takes the form of: http://<server name>/<instance name>/XRMServices/2011/Organization.svc
Right-click the BizTalk project and choose Add and then Add Generated Items. Select Consume WCF Service. On the Metadata source page of the BizTalk WCF Service Consuming Wizard, choose the source of the metadata to be the Metadata Exchange (MEX) endpoint.&; On the Metadata Endpoint page of the wizard, enter the URL of your Dynamics CRM SOAP endpoint. After clicking the Get button next to the Metadata Address (URL), move to the next page of the wizard and click Import.
The wizard adds a host of files to the BizTalk project. This includes two binding files, an orchestration (that includes message and orchestration port definitions), and seven schemas that represent the data types used by this service.
Unfortunately, the schemas generated for this endpoint are not considered valid. They do not have the proper links between each other and an error is shown when you open one. You could either manually correct each schema, or, better yet, leverage the valid BizTalk schemas included in the Dynamics CRM SDK that can be downloaded from the Microsoft website.
Delete the seven XSD schemas added by the service reference.
Right click the BizTalk project and choose Add then Existing Item. Navigate to where you have unpacked the Dynamics CRM SDK package and find the sdk\schemas\crmbiztalkintegration folder and choose the seven "organizationservice" schemas and click Add.
Unfortunately, the orchestration that was generated by the Dynamics CRM service reference will now contain errors due to small changes in the type names of the new schemas compared to the generated ones. So why go through the effort of adding the service reference at all if we are just going to replace all the schemas and end up with a broken orchestration? One benefit of still adding the service reference is acquiring the binding files. The Dynamics CRM endpoint has specific configurations that do not exactly match the standard BizTalk WCF bindings. These binding files make it simpler to get our Dynamics CRM send port correct the first time. Note that in real life, it is best to build a single project that contains the Dynamics CRM schemas and reference it in future projects.
Exclude the generated orchestration from the BizTalk project and build the project to confirm that the added schemas are valid.
Right-click the BizTalk project and choose to add a new item. Select the Map type and name it Customer_To_CrmContactQuery.btm and click Add.
For the source schema, select the Chapter3-DynamicsCRM.AccountInt.Customer_XML message type. This map takes in the Customer message type which initially only contains a value in the ID field. When a response is received from Dynamics CRM, we will populate the rest of the Customer message's nodes.
For the destination schema, select the Chapter3-DynamicsCRM.AccountInt.organizationservice_schemas_microsoft_com_xrm_2011_contracts_services message type. When prompted, select Retrieve as the root node. Our map now looks like the following image:
16 The Retrieve schema for Dynamics CRM is untyped, as mentioned earlier. Provide the name of the entity being queried (entityName), the unique ID of the target record (id), and either a list of which specific attributes we want (Columns) or request every attribute of the record (AllColumns).
We could hard-code the entityName value&;&; by setting the value attribute of the node from the Properties window of the BizTalk Mapper, but those settings can be difficult to locate later on. One way to visually indicate a default node value is through a functoid. From the Visual Studio 2010 toolbox, open the Advanced Functoids tab&; and drag a Value Mapping functoid to the mapping grid. Open the functoid by double-clicking it and set the value of the first parameter to true (so that it always applies) and set the Result value to contact (which is the formal name of the Dynamics CRM entity). Finally, drag the output of the functoid to the entityName node in the destination schema.
As mentioned above, the only field in the source schema that initially contains a value is the ID field. Map that source field to the id field in the destination. Note that because the destination fieldid has a pattern attached to it that looks for a GUID format, set the Value property of the source node to a valid GUID so that the map can be tested successfully. Do not worry about introducing side effects at runtime as any Value set on a source schema node is ignored outside of Visual Studio 2010. However, be aware that setting the Value property&; of a destination schema node does result in that value being pushed to the mapped message at runtime.
There are 146 fields in the Dynamics CRM contact object, so choosing to return all columns may be excessive. If you do want to retrieve all columns, simply set the destination schema's AllColumns node to true. However, in this case, we will only request specific columns. To generate the repeating elements under the Columns node, write some custom XSLT. From the Visual Studio 2010 toolbox, under Advanced Functoids, drag the Scripting functoid to the mapping grid.

Double-click the functoid and switch to the Script Functoid Configuration tab. Choose the Inline XSLT option&; since there is no source schema input to this custom XSLT and the objective is to emit a set of XML elements. Enter the following XML to retrieve just the desired columns. Notice the explicit namespace declaration to properly "type" the string and prevent errors during service invocation:

<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">salutation</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">firstname</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">middlename</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">lastname</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">address1_addresstypecode</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">address1_line1</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">address1_city</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">address1_stateorprovince</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">address1_postalcode</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">telephone1</string>
<string xmlns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">emailaddress1</string>

Save and close the map.
Right-click the BizTalk project and add a new item. Select the Map type, set the name to CrmContact_To_Customer.btm.
For the source schema, select the Chapter3-DynamicsCRM.AccountInt.organizationservice_schemas_microsoft_com_xrm_2011_contracts_services type and choose RetrieveResponse as the root.
For the destination schema, select the Chapter3-DynamicsCRM.AccountIntCustomer_XML message type. The resulting map will look like this:
The message coming back from Dynamics CRM (which is the source of this map) contains an array of fields and values for the returned entity. The Attributes node&; contains all of the entity values. When there is an option set, or list of values, involved (e.g. addresstypecode) the Attributes value would be "1" while the FormattedValues node, which contains friendlier representations for option set values, would have an entry for addresstypecodewhere the value would be "Default Value".
To map the repeating KeyValuePairOfstringanyType to the structured destination fields, we can either build one long custom XSLT block, individual XSLT blocks for each section, or try and use other functoids (for example, Logical functoids with Value Mapping) to match the repeating source nodes to the destination node. In this example, build individual blocks of XSLT necessary to populate the entire destination schema.
Drag a Scripting functoid to the mapping surface and set it to use Inline XSLT. This functoid connects to the destination ID node and has no link from a source field. The XSLT for this functoid looks for the value node associated with the key node that holds the contactid value.
Code View: Scroll / Show All
```
<ID>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='contactid']/*[local-name()='value']"/>
</ID>


					  
```

Next, drag another Scripting functoid to the mapping surface and set it to also use Inline XSLT. Connect it to the Name destination node. Much like the previous functoid, it will find the value nodes corresponding to the key nodes for each element under the Name. Highlighted here are the portions of the XSLT that retrieve the requested values.

<Name>
<Prefix>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='salutation']/*[local-name()='value']"/>
</Prefix>
<FirstName>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='firstname']/*[local-name()='value']"/>
</FirstName>
<MiddleName>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='middlename']/*[local-name()='value']"/>
</MiddleName>
<LastName>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='lastname']/*[local-name()='value']"/>
</LastName>
</Name>

Add another Scripting functoid that uses Inline XSLT and connects to the destination node named Addresses:

<Addresses>
<Address>&;&;
<Type>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringstring'][*[local-name()='key']='address1_addresstypecode']/*[local-name()='value']"/>
</Type>
<Street>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='address1_line1']/*[local-name()='value']"/>
</Street>
<City>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='address1_city']/*[local-name()='value']"/>
</City>
<State>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='address1_stateorprovince']/*[local-name()='value']"/>
</State>
<PostalCode>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='address1_postalcode']/*[local-name()='value']"/>
</PostalCode>
</Address>
</Addresses>

Next, add a Scripting functoid that leverages Inline XSLT and connects to the PhoneNumbers node:

<PhoneNumbers>
<PhoneNumber>
<Type>Business</Type>
<Number>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='telephone1']/*[local-name()='value']"/>
</Number>
</PhoneNumber>
</PhoneNumbers>&;&;

Finally, add one more Scripting functoid that uses Inline XSLT and connects to the EmailAddress node in the destination schema:

<EmailAddress>
<xsl:value-of select="//*[local-name()='KeyValuePairOfstringanyType'][*[local-name()='key']='emailaddress1']/*[local-name()='value']"/>
</EmailAddress>

The final map looks like the following image:
This service will be consumed by a BizTalk orchestration, so right-click the BizTalk project and add a new item. Select orchestration and set the name to QueryCrmCustomer.odx. This solution could consume the service directly through messaging (without orchestration), but because the generated BizTalk send port will have multiple possible operations to invoke (for example, Create, Retrieve, Update), an orchestration is the easiest way to set that target operation.
Create four orchestration messages: Customer_Input of type Chapter3-DynamicsCRM.AccountInt.Customer_XML, Customer_Output of type Chapter3-DynamicsCRM.AccountInt.Customer_XML, ContactRetrieve_Request of type Chapter3-DynamicsCRM.AccountInt.organizationservice_schemas_microsoft_com_xrm_2011_contracts_services.Retrieve and ContactRetrieve_Response of type Chapter3-DynamicsCRM.AccountInt.organizationservice_schemas_microsoft_com_xrm_2011_contracts_services.RetrieveResponse.
Sketch out an orchestration flow that receives a message, transforms a message, sends a request to a service, receives a response, transforms the response, and sends out a final message. The top Receive shape, named Receive Request, is associated with the Customer_Input message. The Construct shape, named Construct Query, builds the ContactRetrieve_Request message. The Transform shape uses the Customer_To_CrmContactQuery map that was built earlier. The next Send shape, named Send Request, uses the ContactRetrieve_Request message while the Receive shape that follows, named Receive Result, gets back the ContactRetrieve_Response message&;. The next Construct shape, named Construct Output, builds the Customer_Output message. Its Transform shape uses the CrmContact_To_Customer map where the Transform destination is set to Customer_Output. Finally, the last Send shape, named Send Result is tied to the Customer_Output message.
Add a one-way orchestration port that receives messages and is tied to the Receive Request shape.
Add a one-way orchestration port that sends messages and connects to the Send Result shape.
Create a two-way orchestration port that sends a message and receives a response. Be sure to set the Operation name of this port to Retrieve. The operation name here must correspond to the operation name of the Dynamics CRM web service method that is being called.
Build and deploy the BizTalk project.