Windows XP
Windows Vista
Windows 7
Windows Azure
Windows Server
Windows Phone
Windows Server

Microsoft Dynamics CRM 4 : Scribe Integration - Scribe Insight Architecture

5/23/2013 5:51:19 PM

Scribe Insight is designed to support the effective deployment of a number of market-leading business applications, including Microsoft Dynamics CRM, Microsoft Dynamics GP, Microsoft Dynamics NAV, Salesforce, and SalesLogix.

Scribe Insight is the core technology that forms the basis for the migration and integration solutions using a unique and open template model that enables companies to quickly and efficiently configure any data integration or migration to meet their specific needs, all without having to write a single line of code. (See Figure 1.)

Figure 1. Data migration and integration.

Because each customer configures their business applications differently and has different requirements for how the integration itself will work, Scribe Insight is designed to provide highly functional integration solutions that meet the specific needs of each customer.

Scribe Insight has five major design points:

  • No programming required: The tool has a graphical user interface that enables business or data analysts (the people who know the issue best) to design and deploy sophisticated integration solutions.

  • A single point of management: Companies can support and maintain the integration solution after it has been deployed.

  • A consistent adapter model: The core Insight design environment views all applications in the same way, while presenting to the designer information about each application that is important to the integration task.

  • Open connectivity: Lets companies integrate these core business systems with the wide variety of applications and data stores that are unique to their business.

  • A template model: Users can quickly assemble reusable integration components and configure them for each deployment’s unique needs. After the initial deployment, required changes in the integration can be accommodated with a simple reconfiguration.

The core components of Scribe Insight are built using the Microsoft Visual Studio development platform for the Windows family of operating systems. The Scribe Server is the core engine that provides connectivity to the various applications, databases, and messaging systems within the integration environment. Communications between the Scribe components and the applications being integrated is provided using the appropriate technology. For example, Scribe adapters to those applications that support web services, such as Salesforce or Microsoft Dynamics CRM, use Simple Object Access Protocol (SOAP), whereas other on-premise APIs are worked with using Component/Distributed Component Object Model (COM\DCOM).

Figure 2 shows the topology of Scribe Insight components. These items represent the five major Scribe Insight components:

  • Scribe Server

  • Scribe Workbench

  • Scribe Console

  • Adapters

  • Templates

Figure 2. The Scribe topology.

Scribe Insight is based on a loosely coupled, yet tightly integrated architecture that is highly adaptable to each customer’s unique and constantly changing business environment. For example, each adapter communicates to the Scribe Server in precisely the same way regardless of the application or database to which it is connecting. This abstraction of the application or database details provides for a highly productive design environment; once users learn to use the Workbench, they can design integrations with a wide variety of applications and data stores. This abstraction also means that templates (representing specific integration processes between applications or databases) are insulated from most changes/updates to the application or database interface. The same template that works with version x of an application will continue to work with version y, requiring no reconfiguration except to accommodate substantive changes in the schema or functionality of that application.

The Scribe Server

The Scribe Server is the core of Scribe Insight–supported integration processes and facilitates the exchange of data between two or more applications or databases. Because Scribe Insight, in essence, brokers a conversation between these applications and databases, it can support a highly heterogeneous server environment of Windows, UNIX, Linux, on-demand applications, and so on. All that it requires is a “connection” to these applications via a Windows client, a non-platform-specific middleware protocol such as Open Database Connectivity (ODBC), or via a Microsoft Message Queuing (MSMQ) message queue.

Underlying the Scribe Server are a number of Windows services designed to monitor and detect events, process messages, raise alerts, and provide an access point for the Scribe Console to the other services. The Scribe Server also includes its own internal database that stores all execution and error logging, persisted integration settings, cross-reference tables, and important integration statistics. The Scribe internal database can be configured to support the Microsoft SQL Server Express database (provided with Scribe Insight) or the other Microsoft SQL Server editions.

The Scribe Workbench

The Scribe Workbench provides a rich graphical environment where users design and configure the data mappings and business rules that define the integration solution. All work completed in the Scribe Workbench is “saved” in a lightweight file that is referenced by the Scribe Server at runtime. This self-documenting, metadata-driven model allows for easy debugging during the deployment phase and rapid modification as the application environment or business needs change.

The Scribe Workbench enables you to connect to your applications, define a source result set, configure object-level target processing, and then just point and click to modify or add data mappings.

One of the key capabilities in the Scribe Workbench is the ability to “normalize” source data on-the-fly as it processes against the target application. In other words, single- or multi-row “source” data can have multiple operations executed per row on “target” data objects. These operations, referred to as steps, can be conditionally executed based on user-defined logic, allowing complex, transaction-enabled, multiple-object operations. (See Figure 3)

Figure 3. The Scribe Workbench.

With the Scribe Workbench, designing complex data transformations is a simple task. The Scribe Workbench provides more than 150 Microsoft Excel-like functions for data translation, including the following:

  • Parsing functions for names and addresses

  • Date and time conversions

  • String manipulation

  • Database and file lookups for processing synonym values

  • Logical if/then/else functions to support conditional processing

In the rare case where there is a need for data transformation beyond what is included in the Scribe Workbench, additional functions may be created using COM and simply added to the function list.

The Scribe Workbench was designed to support many advanced integration needs beyond data transformation and mapping and includes these additional capabilities:

  • A Test window that shows the results of processing test data without committing the data to the target system. Users can view the results of data translations and object-level transaction processing for easy and efficient debugging of integration processes.

  • Built-in system key cross reference and management, designed to dynamically maintain data integrity between records across two or more loosely coupled applications.

  • Built-in support for foreign key value reference management, designed to dynamically maintain data integrity between related records within an application.

  • Net change tracking by updating or deleting successfully processed source records or by comparing a source-side update stamp against a variable last run date/time in the source query.

  • Conflict detection and resolution to support bidirectional data synchronization.

  • Formula-based lookups for “fuzzy” record matching logic.

  • Value cross-reference and lookup support.

  • Automatic data type mismatch resolution.

  • Transactional support for Header-Detail type data sets.

  • Configuration of target-side commit and rollback.

  • Rich error handling and logical flow control, including support for user-defined errors.

  • Rejected row logging to support automated repair and recovery processes.

The Scribe Console

The Scribe Console is a Microsoft Management Console snap-in that provides the user interface to an array of powerful features used to set up, organize, and manage key aspects of any number of integration processes. The Scribe Console is the main user interface to the capabilities underlying the Scribe Server.

The Scribe Console can be installed independently from the Scribe Workbench, and can be configured to connect to the Scribe Server using either COM/DCOM technology over a LAN or a provided SOAP-based web service connection hosted by Microsoft Internet Information Services (IIS).

The Scribe Console (shown in Figure 4) provides a single point of management for a company’s various integration points, organizing them as discrete units of work or collaborations. Each collaboration is a series of related integration processes and instructions for how and when these processes should be automatically executed. Collaborations are organized in a graphical, user-defined tree and can be managed as independent objects with their own reporting, monitoring, and diagnostic functions. The Scribe Console also provides easy access and control of all integration processes running on the system, through controls implemented at the integration server level.

Figure 4. The Scribe Console.

At the core of the Scribe Console are its sophisticated event management capabilities. The Scribe Console allows each company to precisely define the proper latency for each integration process, from scheduling batch processes to run on a predefined time period, to establishing near real-time polling intervals based on a file drop in a directory or the results of a source-side query, to the real-time processing of messages arriving in an in queue.

The Scribe Server is built with a modular, multithreaded architecture that allows for scaling of integration processes based on the available CPU processing strength. It also features efficient connection sharing to maximize performance, where possible.

Additional capabilities of the Scribe Console include the following:

  • Access to the files on the Scribe Server that may need to be moved, copied, renamed, or deleted.

  • Automated system monitoring of business-level events or integration errors with configurable alerts via email, page, and net send notification.

  • For those data sources that do not have a built-in net change mechanism (including event-based message publishing, time and date stamps for updates, or other forms of update stamps), the Console provides a Query Publisher that compares time-based “snapshots” of a source system and publishes the differences as an XML message.

  • Settings to launch an executable file to run before or after an integration process. One example where this pre- or post-execution processing can be useful is the ability to move files into an archive directory after the process is executed.

  • Onscreen editable views of predefined queries that can be displayed in chart or list format.

  • User interface for MSMQ management providing message viewing, moving, copying, and deleting.

  • Review execution history of what processes succeeded or failed, including detailed error reporting.

Scribe Adapters

Scribe adapters enable Scribe Insight to communicate seamlessly to a variety of applications, databases, messages, and files.

Scribe adapters present rich levels of schema information, which is available through the application interface (via a “declare” type function in an application API or in a Web Service Definition Language [WSDL] in the case of web services interfaces) to the Scribe Server and Scribe Workbench. This schema information includes object properties and interrelationships and detailed field attributes such as labels, data types, lengths, restrictions, and default/pick-list values. Combined with the rich features in the Scribe Workbench, this information provides for unparalleled control over integration processes and eliminates the “last-mile” coding required with other integration tools.

Scribe adapters are classified in two ways: enterprise application adapters and connectivity adapters.

Enterprise Application Adapters

Enterprise application adapters are adapters that have been designed and developed to optimize Insight for use with Scribe-targeted CRM and ERP applications, including Microsoft Dynamics CRM, Microsoft Dynamics GP, Microsoft Dynamics NAV, Microsoft Dynamics AX, Salesforce, and SalesLogix. Scribe’s enterprise application adapters are sold as add-ons to Scribe Insight. Key features of these application adapters include the following:

  • The automation of common data loading tasks such as assigning primary ID values, setting default values, validating input data, and setting object relationships—all designed to eliminate runtime errors and provide for greater data integrity.

  • Dynamic discovery that presents the unique configuration of each application or database instance to the Scribe Console and Scribe Workbench at runtime and adjusts to changes in the application or database schema without requiring recoding or recompiling.

  • The seamless integration of application and database error messages to provide detailed exception reporting and handling from the Scribe Console’s single point of management.

Connectivity Adapters

Connectivity adapters are included in core Scribe Insight product. They are designed to complement the enterprise application adapters by providing a wide variety of integration options to support connectivity to the varied applications and data stores within each company’s computing environment. These connectivity adapters enable Scribe Insight to communicate with applications and databases in the following ways:

  • Direct communication with database tables, views, and stored procedures through ODBC 3.0 or later and natively to SQL Server. Scribe leverages all the filtering and querying capability of these databases when employing this approach.

  • The exchange of flat files or XML documents via a directory or FTP/HTTP location.

  • The asynchronous exchange of XML messages via an industry-standard message queue, email, or integration broker.

  • SOAP Messages via web-based transport protocols, such as http/https.

A common use of Scribe’s connectivity adapters is to support integration between the targeted applications served by Scribe’s application adapters and a wide variety of other packaged enterprise applications. These other enterprise applications include the following:

  • ERP and CRM systems from SAP, Siebel, Oracle (Oracle, PeopleSoft, JD Edwards,) Sage (MAS 90/200/500,) Epicor, and so on

  • Packaged applications that serve a particular niche or vertical market

  • Custom in-house-developed systems

Scribe provides a number of approaches to integrating with these applications, depending on the business requirements and available technical resources, including the following:

  • Directly to the database: This is a simple, straightforward approach if you are migrating from an application or your project is limited to a one-way feed of data from that application. Scribe Insight provides a number of methods using this approach to extract “net change” data from the application.

  • Via interface tables: Many applications support a set of interface or staging tables that provide for a safe way to integrate data into that application. After data is passed into the interface tables, an application process is initiated that validates the data and applies appropriate application rules. With Scribe Insight, you can write to these tables and initiate the application process automatically.

  • Via an XML/messaging interface: Many enterprise applications provide an XML interface that is incorporated into the workflow engine within the application. Using this method, Scribe Insight can publish XML messages into a message queue for real-time integration with the other application. Scribe Insight can also receive XML transactions published by the application’s workflow engine into a message queue in real time.

  • Via the application’s API: Many applications expose a web services- or COM-based API where transactions can be passed to the application. Data can also be queried via this API. Out of the box, Scribe Insight cannot “natively” integrate with this API; however, custom code can be written to convert these calls into an intermediate format. This intermediate format can be an XML message, a flat file, or a record in a database staging table.

Scribe Insight also includes connectivity adapters for data migration from/to certain leading desktop applications, including ACT!, GoldMine, and Microsoft Outlook/Exchange.

Scribe Templates

Scribe templates represent complete or partial data integration or migration processes that have been developed using Scribe Insight technology. Scribe provides a number of these templates as free downloads from the Scribe Web Community to support the successful deployment of Scribe Insight.

Templates consist of the building blocks of a fully functional migration or integration solution as configured with Scribe Insight, including the following:

  • Source-side “net change” processes and filtering

  • Event and process automation

  • Data mappings

  • Record matching for updates and duplicate avoidance

  • User/owner mappings

  • Field ownership and update rules

  • System key cross referencing and management

  • Connection validation and security

  • Data ownership and customizations

  • Application customizations

  • Transaction management

  • Commit and rollback settings

  • System monitors and alerts

  • Business monitors and alerts

There are two distinct styles of Scribe templates: solution templates and component templates.

Solution Templates

Solution templates represent a complete, fully functional integration or migration solution between two applications. Examples of these include migration solutions for ACT! into Microsoft Dynamics CRM, SalesLogix, or Salesforce and “front to back-office” integration solutions between Microsoft Dynamics GP and Microsoft Dynamics CRM or Salesforce. Scribe’s unique template model provides out-of-the-box functionality for these integration scenarios, built over its industry-leading integration tool. Because most customers have business needs unique to them, these standard templates can be quickly extended and customized via the GUI-based mapping and development environment.

The component architecture of these solution templates also enables customers to implement templates in phases or pick and choose the elements of the templates that they require. In the front to back-office integration example, a customer might not want to implement order integration initially (or in some cases never) but can still synchronize customer activity (accounts, contacts, invoices) between the two systems. This modularity enables customers to implement an integration solution tailored to their exact needs.

Component Templates

Component templates are starting points for common integration processes used by customers that are implementing an integration solution for which Scribe has not developed a solution template.

For example, Scribe provides templates that integrate customers, products, orders, and invoices between a Scribe-developed sample ERP system and Scribe’s targeted CRM applications, including Microsoft Dynamics CRM, Salesforce, and SalesLogix. A customer that is looking to integrate one of these applications with their own ERP application can use the appropriate component template as a significant starting point. Typically, these component templates provide the bulk of the end solution, with the remainder easily configurable with Scribe Insight.

Other -----------------
- Microsoft Dynamics CRM 4 : Scribe Integration - Integration Options
- Maintaining Dynamics GP : Safeguarding data by Backing Up everything
- Maintaining Dynamics GP : Improving stability by Managing Dictionaries
- Client Access to Exchange Server 2007 : Using Cached Exchange Mode for Offline Functionality
- Client Access to Exchange Server 2007 : Using Outlook 2007 Collaboratively (part 3) - Using Group Schedules
- Client Access to Exchange Server 2007 : Using Outlook 2007 Collaboratively (part 2) - Sharing Information with Users Outside the Company
- Client Access to Exchange Server 2007 : Using Outlook 2007 Collaboratively (part 1)
- Windows Server 2003 on HP ProLiant Servers : The Physical Design and Developing the Pilot - Time Services (part 2) - Domain Time Hierarchy
- Windows Server 2003 on HP ProLiant Servers : The Physical Design and Developing the Pilot - Time Services (part 1) - Time Services Role in Authentication
- Windows Server 2003 on HP ProLiant Servers : The Physical Design and Developing the Pilot - Network Services
- First look: Apple Watch

- 10 Amazing Tools You Should Be Using with Dropbox
- How to create your first Swimlane Diagram or Cross-Functional Flowchart Diagram by using Microsoft Visio 2010 (Part 1)

- How to create your first Swimlane Diagram or Cross-Functional Flowchart Diagram by using Microsoft Visio 2010 (Part 2)

- How to create your first Swimlane Diagram or Cross-Functional Flowchart Diagram by using Microsoft Visio 2010 (Part 3)
Popular tags
Microsoft Access Microsoft Excel Microsoft OneNote Microsoft PowerPoint Microsoft Project Microsoft Visio Microsoft Word Active Directory Biztalk Exchange Server Microsoft LynC Server Microsoft Dynamic Sharepoint Sql Server Windows Server 2008 Windows Server 2012 Windows 7 Windows 8 Adobe Indesign Adobe Flash Professional Dreamweaver Adobe Illustrator Adobe After Effects Adobe Photoshop Adobe Fireworks Adobe Flash Catalyst Corel Painter X CorelDRAW X5 CorelDraw 10 QuarkXPress 8 windows Phone 7 windows Phone 8 BlackBerry Android Ipad Iphone iOS
Popular keywords
HOW TO Swimlane in Visio Visio sort key Pen and Touch Creating groups in Windows Server Raid in Windows Server Exchange 2010 maintenance Exchange server mail enabled groups Debugging Tools Collaborating
Top 10
- Microsoft Excel : How to Use the VLookUp Function
- Fix and Tweak Graphics and Video (part 3) : How to Fix : My Screen Is Sluggish - Adjust Hardware Acceleration
- Fix and Tweak Graphics and Video (part 2) : How to Fix : Text on My Screen Is Too Small
- Fix and Tweak Graphics and Video (part 1) : How to Fix : Adjust the Resolution
- Windows Phone 8 Apps : Camera (part 4) - Adjusting Video Settings, Using the Video Light
- Windows Phone 8 Apps : Camera (part 3) - Using the Front Camera, Activating Video Mode
- Windows Phone 8 Apps : Camera (part 2) - Controlling the Camera’s Flash, Changing the Camera’s Behavior with Lenses
- Windows Phone 8 Apps : Camera (part 1) - Adjusting Photo Settings
- MDT's Client Wizard : Package Properties
- MDT's Client Wizard : Driver Properties
Windows XP
Windows Vista
Windows 7
Windows Azure
Windows Server
Windows Phone
2015 Camaro