↧
Larry Cone to Speak at Boomi World 17 on Boomi and Master Data Management
↧
Boomi World Thanks!
Boomi World 17 - The first Dell-Boomi Customer Conference has been concluded, and I’m recording impressions and events. The first impression is high quality – even though this was the first event, and the planing and execution was done in four months, the overall fit and finish was great. Consistent, high quality signage and branding; a quality AV/show production team for the live sessions, and a nice partner expo set up that fit the space and the traffic. The expo felt busy and active without seeming claustrophobic.
↧
↧
Three Principles of Data Integration Design – Empty Pipe
I have recently had a chance to speak with lots of Boomi customers. Boomi World 17 was an opportunity for us at Kitepipe to connect with our customers, and the larger Boomi community. Often these discussions very quickly got to best practices, and design approaches. Here are three principles that I apply in my integration projects
↧
Three Principles of Data Integration Design – Empty Pipe
I have recently had a chance to speak with lots of Boomi customers. Boomi World 17 was an opportunity for us at Kitepipe to connect with our customers, and the larger Boomi community. Often these discussions very quickly got to best practices, and design approaches. Here are three principles that I apply in my integration projects
↧
Three Principles of Data Integration Design – No Black Hole
I’m covering three design principles we use at Kitepipe. These are:
- Empty Pipe
- No Black Hole
- Part of the Business Process
This post will consider No Black Hole.
No Black Hole refers to the widespread practice of thinking about integration as “data synchronization”, and making it invisible to the user. Making it a Black Hole.
Much of what we do at Kitepipe is transaction integration, not data synchronization. In transaction integration, the payload has high value - and somebody, maybe many somebodies, care if the integration completes successfully. These might be Sales orders, shipments, or employee status updates. So we work hard to make the results visible to the user. In all cases we notify about errors, and often about successful integrations, as well.
We use a range of techniques to communicate to users about the status of integrations. Techniques include:
- emails that alert to errors
- emails that alert to success
- logging to a log database or application
- posting status back to the source application
Our experience is that users are very interested in integration status, and welcome “rich information” about the status of the transactions. Giving users prompt, easy to access status information also has another important benefit - it takes the IT or Data or Integration team out of the data correction loop. Users can see data problems, fix them, and rerun them.
But only if the integration is part of the overall business process - which is the next principle.
↧
↧
Looking for Boomi training?
A frequently asked question to Kitepipe is where and how to get Boomi Training. The answer is that it is free and easy.
↧
Cloud-Based architecture and Integration : Notes from the front lines
I’ve had the privilege of being on the front lines of the move from on-premise based to cloud-based enterprise application architectures. For the last five years I’ve run Kitepipe, a cloud integration services firm, and participated in a wide range of projects involving the move of IT infrastructure, applications, and business functions to the cloud.
In the following series I’ll share my experiences and observations on this fundamental shift in application architecture. I’ll focus on the integration implications, as that is my practice area. Kitepipe gets involved when the integration implications of moving a business process from an on-prem location to a cloud application become evident. For some this is in the planning/sales process. Sometimes this is during the implementation process. And quite often integration needs only become evident after the move, when the pain/opportunity equation of implementing integration becomes evident.
Lets first distinguish between two types of migration to the cloud - Applications in the cloud, and infrastructure in the cloud.
From Amazon, you can rent servers in their cloud. These are large physical servers in a server farm operated by Amazon, which are subdivided into virtual servers that can be created at will. These virtual machines (vms) function just like a server on your site. You can get them running windows, or a range of Linux variants, with a wide range of resource specs in terms of RAM, storage, and network bandwidth. Firms other than Amazon Web Services (AWS) provide this service, but AWS is the largest and best known.
Your company rents a set of these from AWS, and moves jobs, databases, and applications from on-site servers in your data center to these AWS servers. Once network connectivity is established, things run just the same as they did on-premise except that you no longer own and operate the hardware. This is infrastructure in the cloud, or IaaS (Infrastructure as a Service). Its a service because you don’t own it, but you rent it “as a service”.
But, the software applications that run on it are yours, or purchased/licensed by you. That part hasn’t changed. You may still be running your licensed version of SAP, but now the hardware it is running on is owned by Amazon, and rented by you. And the application architecture, and integration architecture, is still the same. Once network connectivity is in place, however you were getting data in and out of those applications has not changed.
In contrast, the rise of cloud applications, or SaaS (Software as a Service) has changed the architecture and integration landscape. In SaaS, you are no longer running “your” application, code, or database, but renting a seat who can use the vendor’s fully managed application to execute a specific business process for your company. The biggest and best known SaaS vendor is of course, Salesforce. They started with an application focused on a very common use case - that of managing the sales process and its associated data. This application had two important innovations:
- Multi-tennancy - all data for all users is in the same set of data base tables
- Sold as a service - the customer rents the use of the application per month per user
These two innovations, working together, completely transformed the global $400B enterprise software industry (Gartner, 2107). I’ve written in detail about the hows and whys of this transformation, here and here. The combination of dramatic cost reduction, dramatic feature increase, and removal of buying and implementation barriers made SaaS applications the overwhelming choice for new implementations, and increasingly for replacement implementations of business application software.
In contrast to IaaS, SaaS has changed the architecture and integration landscape, in several important ways. The first way is obvious - you are running your business process on an application and data structure that you don’t own (you just rent it) and you typically can’t access the data structures directly. Thus the rise of the API (application programming interface), which gives you a programmatic interface (structured calls and returns) to your data base. This is different from on-premise applications, where you typically had access to the data structures, and made data base queries to get bulk data that you moved to another data base.
The other impact of SaaS is more subtle - the rise of “Best of Breed” applications, and the fragmenting of the application software space into sub and sub-sub domains. For a number of reasons, including ease of penetrating the market, SaaS applications have focused on a specific business function for a specific market segment. This has lead to an explosion of SaaS applications on the market. No one knows how many, but informed estimates put the number at well over 10,000. No longer do you buy an HR application, you license use of a recruiting and on boarding application for tech companies (such as Jobvite - a bit of googling found a SaaS solution in this randomly chosen segment).
this explosion means that an enterprise no longer has a dozen applications on-premise, but now has fifty applications in the cloud - each one the best of breed for a specific business process (on-boarding, lease accounting, community management, etc). The compelling economics of best-of-breed SaaS applications means that a department can buy their own solution, without central IT support. In fact, some of our fast growing tech customers have no central IT function in the traditional sense, but each functional group sources their own applications.
This proliferation of cloud applications blows away traditional notions of IT architecture and master data management. The cows are out of the barn, IT can just chase them around the pasture. How do you define and manage an overall information architecture when your departments and business units acquire and set up their own applications?
As the application landscape fragments and moves to the cloud, so the business process data created by customers using these applications moves to the cloud as well. This is everything from a job applicant’s address to the pricing of a widget you just sold. This data is now all over the place, in 50 different cloud data bases that you can only access through an API, or programming interface. But, the HR Benefits application needs the addresses from the recruiting application. And the billing application needs the pricing from the sales application. How is all this valuable business data, spread across fifty application platforms, going to move from one to another to enable seamless business processes?
Well, you need integration tools. You need structured means of moving business process data from one cloud application to another.
Enter IPaaS, or Integration Platform as a Service. About 10 years ago, a very smart guy from Philadelphia, Rick Nucci, saw that all of the above would come to pass, and that there would be a need for integration tools to connect this coming blizzard of cloud applications. So he bet the farm on converting his on-premise EDI software company, Boomi, to the first cloud-focused integration platform, or a segment that would become IPaaS. The Boomi integration was the first, and remains the best set of integration tools for the new “cloud-fragmented” application architecture. Boomi has the important features needed to be a complete solution to the new cloud application architecture, including:
- Huge library of Connectors that talk to the Application APIs
- High performance ATOM runtime engine on premise, or use the Boomi ATOM in the cloud
- Codeless, hyper-productive builder tools that allow you to build and change integrations quickly
- Full SDLC support including build, deploy and monitoring tools
- Expanded product line includes API management, MDM, Queues and more
We will talk more about how this most mature Boomi cloud application tool set fits the needs of the cloud integration specialist.
So, there is a need for enterprises to integrate this new cloud-fragmented application space. But what do these integration architectures look like? In the past there have been a series of integration approaches that moved in and out of favor - including enterprise data models, enterprise data warehouse, and enterprise service bus. What is the emerging integration architecture in the cloud-fragmented application space? And who would be knowledgable about this emerging IT practice area?
Enter Kitepipe, a Cloud Integration Services firm, specializing in building integration processes in the emerging cloud-fragmented application space. Kitepipe’s founder, Larry Cone (your humble author) is another smart Philly guy. I was VP Engineering at an early cloud application company near Philadelphia. So early that we didn’t call it Cloud, or even SaaS, but an ASP, Application Service Provider. We needed an integration tool to connect to our newly-installed Netsuite instance, and I selected Boomi as our integration tool. I knew about Boomi and Rick through the local IT ecosystem, and they were a natural choice. I took the training, and did some hands-on building with the Boomi tools, and I really liked the product.
My insight was a bit later. I too saw the coming fragmentation of the application space into the cloud, and wrote about it here and here. I too saw the coming need for integration between these cloud based applications. I too saw that Boomi was a great toolset, purpose-built for this role. I started a cloud integration services company, Kitepipe, to address this coming market. Kitepipe is today a leader in implementing and managing Boomi integration suites for companies large and small.
And I had one additional insight, after working at integration for a while, which is this: That the value to the business comes from thinking about integration at the business process level, rather than the data level. So, don’t think about moving closed-won opportunities from Salesforce to Netsuite - think about how the integration process can complete the sales-fulfillment-billing cycle across multiple applications. Understand the operational challenges, and build an integration process that solves those challenges.
But what does the integration architecture look like? At last, we are getting to the core of this series of posts. Lets take that sales process as an example, deconstruct it, and look at the architecture that results from implementing using best practices.
Often, the best way to proceed is to understand the set of problems to be solved. I call this "selling to pain” - its the approach I use in the sales process to convey the value of the proposed integration project. so, here are some typical pain points in the sales to accounting integration:
- different field and data structures
- different code values for the same data
- wrong customer
- wrong/bad pricing
- special or promotional pricing
- wrong territory or geography
- wrong currency
- related deal components - not SOX compliant
- all happens at the end of the quarter
- not well forecasted
- new products/offerings not yet in accounting
The number one role of the integration is to transform fields and codes from sales format to accounting format, but that is just the start. Most of the above issues can be characterized as “bad data” - data from Sales that does not fit (yet) into Accounting. An effective integration approach needs to start with the understanding that most integration problems are data problems. Based on this, we at Kitepipe have developed a template for building highly functional integrations.
So the number one role of the integration access is access - the process needs to be able to selectively query the data in the right state; the next role is Transform - map source data to target data profiles; The number three role is Enrich - look up or add master data to the transaction, as needed; the next role is Validation - is all the date good? the fifth role is alerting - tell someone who can fix the data. The sixth role is to Post the transaction to the target application, and detect any errors; The seventh role is to Synchronize - make sure that there is a key pair between the application platforms that can be used to match the transactions for future updates. The next role is Recovery - the integration process should be recoverable in case of failure, ideally by just running it again. The last role is Status - make sure users on either side know what happened, and what the status of the transaction(s) are.
So thats:
- Access - select the data you want
- Transformation - map source to target
- Enrich - include data from other sources to enrich the transaction
- Validation - insure that the data is valid
- Alerts - notify those who can fix the bad data
- Post - post the new or updated transaction to the target system(s), and detect any errors
- Synchronization - post keys to target, and back to the source to confirm traceability
- Recovery - structure the process for recovery, ideally by rerunning, and without intervention
- Status - post status to logs, people, source and target platforms
This set of features is built in to our process templates at Kitepipe, and speeds the integration development process. Because Boomi is a powerful purpose-built platform for building integrations, all of the above functionality can be quickly built, tested and deployed, without writing code.
These functions of the integration are architecture at the process level - a Boomi process can (and should) accomplish all of these functions for a given set of transactions. There are additional choices to make which impact how the processes executed - batch vs. real time, for instance - but in Boomi the integration structure and logic is the same, the only difference is how the process is invoked. You can select a batch of records from the sales system and process them, or deploy a “listener” or web service - that accepts calls from a trigger in the Sales system to process sales records on an event basis. But in Boomi the process structure is the same.
This represents a ‘Point to Point” integration solution - from the sales application to the accounting application. Many of the integrations we build at Kitepipe - and we build a lot of integrations - are of this type. - but surely this is not an enterprise integration architecture? Well, yes it is - a point to point integration solves a business process problem quickly and effectively.
But what about my complex enterprise where we have possibly fifty application endpoints, and and the possible point to point integrations is in the hundreds - in fact, Siri tells me that there are 1,250 unique pairs of integration points of 50 applications - twice that if direction is counted separately. This rapidly becomes unmanageable, and a different approach is needed. Or is it…
Having looked at long lists of application platforms in a single enterprise, I can confirm that the vast majority of application pairs, taken randomly, don’t make sense, and would not be implemented. So, you don’t need to manage 1200 integrations. Our most complex customers have implemented 30 or 40 integrations, which is manageable by a small, well-trained integration team. But not all of those are point to point - we use some integration hub techniques to manage complexity in complex enterprises.
Probably the best way to think of integration architecture is based on evolution - what do you do first, and next, and what might the final state look like? This is a good approach to take, because it allows quick wins on the way to a more sustainable architecture.
Ah for the old days, when a corporate IT shop could embark on 24 or 36 month or 5 year re-architecture or master data implementation projects, which got repeatedly delayed, then quietly cancelled. No one has that kind of time or money anymore. We at Kitepipe need to sing for our supper every night, so we better be putting something into production that improves the client's business every six to eight weeks.
So, how do you implement a complex cloud-fragmented application integration architecture? As the old riddle asks, how do you eat the Elephant? One spoonful at a time.
Here is our successful recipe for building a comprehensive cloud integration architecture for the enterprise.
Phase one: Build Rich point to point transaction integrations for the revenue stream.
The revenue stream is where you can have the most impact on most businesses. Speed it up just a little bit, and you pay for the whole party, and make important people happy (like the CFO). Plus the metrics are in place - people will notice if you cut processing time from sale to fulfillment, or reduce the errors between sale and revenue recognition.
This looks different for different businesses - sometimes its customer acquisition, sometimes order processing or rev rec, sometimes fulfillment, sometimes exception handling (returns, back orders, RMAs, etc). But applying integration tools to the pain points in the revenue cycle is always going to make your client a hero!
As a step one A, build additional point to point integrations that support other critical business functions. In some of our high growth customers these are HR or benefits integrations
Phase Two: Build master data hubs for key Master Data objects
As applications proliferate in the cloud, many or most of them need some common master data. Most larger enterprises have the most trouble with customer data. Multiple product lines, sales teams, and divisions have different customer sets, and these are never reconciled. You can often make a revenue case around cross-selling, if only there were a clean, common customer master. This is often a regulatory requirement, or at least a source of embarrassment. One of our projects was funded on the basis that no-one in the organization could agree on how many customers they had. So a customer master data hub is a pain/opportunity point for many organizations.
↧
Kitepipe recognized as leader in Netsuite integration space by CIO Review
In the latest CIO Review, focused on Netsuite solution providers, Kitepipe was recognized as a leader in creating high value integrations using Boomi.
↧
New Video - Why is Kitepipe Unique?
As a Select Services Partner, Kitepipe has a trusted partnership with Dell Boomi. In this new video, Kitepipe CEO Larry Cone talks about:
- Why he began a company that only does Boomi integrations,
- What he feels is so successful about the Boomi platform., and
- How Kitepipe's focus had lead to our success
He dives into the business benefits of a Boomi integrated system and why Boomi will take over the (application) world.
↧
↧
Kitepipe a Diamond Sponsor of Boomi World 2018!
I am pleased to announce that Kitepipe is a Diamond Sponsor of the second Boomi World conference. The Conference is being held at the Wynn Las Vegas on November 5th -7th.
↧
Boomi World 2018 Talk - Lessons from 100 Boomi Projects
I’m Larry Cone, Founder and CEO at Kitepipe. We are a Boomi Select Implementation Services Partner, and we do a lot of Boomi projects - more than 100 over the last two years.
In thinking about what topics would be of interest in my talk at Boomi World this year, I started thinking about the body of work that the Kitepipe team has accomplished, all that we have learned, and the changing integration landscape we see. The evolution of Boomi projects over those engagements has expanded from point to point cloud integrations to more complex business process systems like revenue recognition, employee onboarding, product and procurement supply chain management API management and many more. Our team engages regularly now with the owners of these sub-systems as much as we work with the IT implementers. Boomi has evolved and we have uncovered best practices to make data, transactions and processes interact to transform these businesses with the entire Boomi platform.
My talk, " Lessons from 100 Boomi Projects” will distill that set of project experiences into what we have learned, what best practices have emerged, and how the integration landscape is changing.
My timeslot at Boomi World is from 1:30 – 2:45pm on Wednesday Nov 7th - hope to see you in the audience - I guarantee you will gain fresh insights about Boomi integration from our experiences. Check in at the Kitepipe website after Boomi World for a copy of my presentation.
↧
Lessons Learned- Over 100 Boomi Projects and Counting
Handling more than 100 Boomi projects over the past two years, the team at Kitepipe, a Boomi Select Implementation Services Partner of Dell Boomi, has learned a thing or two about best practices, pitfalls to avoid and trends that are reshaping the integration landscape.
For example, organizations are rapidly expanding Boomi usage beyond straightforward cloud-to-cloud connectivity to cover more complex integration scenarios and business processes like revenue recognition, employee onboarding, product and procurement supply management, and many more.
And the pace of integration has accelerated far beyond what we saw in 2010 when I founded Kitepipe, which today works exclusively with the Boomi platform. It’s a swift and brave new world of connectivity that’s driving faster and smarter business. And Boomi customers are at the forefront of the transformation.
Here are some of the key types of integration projects we are seeing from our customers. These comprehensive projects strategically address a major process workflow from end to end.
1. Full Lifecycle Integrations: Capture all the steps in a transaction lifecycle with a series of integrations. Examples are order-to-cash, revenue cycle transactions, procurement and supply chain.
2. Employee Onboarding: Manage the lifecycle of the employee journey from the system of record (often Workday Human Capital Management or SAP SuccessFactors) as it is mastered and connected to satellite systems.
3. Customer Lifecycle Management With Data Governance: Customer data is often in multiple systems that are critical to your operations. Mastering that data to create “golden records” with Boomi Master Data Hub smoothes out marketing, deal flow, customer support and retention.
4. Better Business Intelligence: Operational integrations drive data consistency, providing a better view of business activity. Improved data accuracy helps leaders trust dashboards, view critical trends earlier and make data-driven decisions.
5. Full-Cycle Workflows: When more processes are automated and integrated, approval flows and real-time user interactions are critical. Key process automation is a requirement for scale and efficiency. Adding Boomi Flow to integrations completes the full integration lifecycle.
What’s also interesting is that our team now engages regularly with business owners of sub-systems supporting those business processes, just as much as we work with the IT team.
That’s a change from the previous IT-centric approach to integration, and it underscores how critical and accessible integration has become to the business. It is the core skill of business users. It is a core skill of the business.
At the same time, integration is now being approached from a strategic perspective, not just tactical.
Integration has become critical to the business while becoming accessible to business users. It is now being approached from a strategic perspective, not just tactical.
Leaders are thinking about how they can proactively rollout strategic integration projects that improve the business compared to reactively deploying integration to fix isolated issues.
And they’re looking beyond transactional data at master data and the value it can bring when properly governed.
The benefits of these trends to C-level executives, business owners and IT professionals are:
- Simpler integration patterns
- Reduction of redundant process and data management
- Data accuracy, consistency and audit trails
- Compliance and regulatory efficiencies
- Real-time interaction and the resulting speed uplift
- Internal stakeholder buy-in across the organization
Our team at Kitepipe is looking forward to an inspiring exchange of ideas, experiences and lessons learned at Boomi World. We’re delighted to have been a Platinum Sponsor of the conference.
↧
Boomi World Thanks!
Boomi World 17 - The first Dell-Boomi Customer Conference has been concluded, and I’m recording impressions and events. The first impression is high quality – even though this was the first event, and the planing and execution was done in four months, the overall fit and finish was great. Consistent, high quality signage and branding; a quality AV/show production team for the live sessions, and a nice partner expo set up that fit the space and the traffic. The expo felt busy and active without seeming claustrophobic.
↧
↧
Three Principles of Data Integration Design – Empty Pipe
I have recently had a chance to speak with lots of Boomi customers. Boomi World 17 was an opportunity for us at Kitepipe to connect with our customers, and the larger Boomi community. Often these discussions very quickly got to best practices, and design approaches. Here are three principles that I apply in my integration projects
↧
Three Principles of Data Integration Design – No Black Hole
I’m covering three design principles we use at Kitepipe. These are:
- Empty Pipe
- No Black Hole
- Part of the Business Process
This post will consider No Black Hole.
No Black Hole refers to the widespread practice of thinking about integration as “data synchronization”, and making it invisible to the user. Making it a Black Hole.
↧
Looking for Boomi training?
A frequently asked question to Kitepipe is where and how to get Boomi Training. The answer is that it is free and easy.
↧
Cloud-Based architecture and Integration : Notes from the front lines
Cloud-Based architecture and Integration - Notes from the front lines
I’ve had the privilege of being on the front lines of the move from on-premise based to cloud-based enterprise application architectures. For the last five years I’ve run Kitepipe, a cloud integration services firm, and participated in a wide range of projects involving the move of IT infrastructure, applications, and business functions to the cloud.
In the following series I’ll share my experiences and observations on this fundamental shift in application architecture. I’ll focus on the integration implications, as that is my practice area. Kitepipe gets involved when the integration implications of moving a business process from an on-prem location to a cloud application become evident. For some this is in the planning/sales process. Sometimes this is during the implementation process. And quite often integration needs only become evident after the move, when the pain/opportunity equation of implementing integration becomes evident.
Two Types of Cloud
Lets first distinguish between two types of migration to the cloud - Applications in the cloud, and infrastructure in the cloud.
From Amazon, you can rent servers in their cloud. These are large physical servers in a server farm operated by Amazon, which are subdivided into virtual servers that can be created at will. These virtual machines (vms) function just like a server on your site. You can get them running windows, or a range of Linux variants, with a wide range of resource specs in terms of RAM, storage, and network bandwidth. Firms other than Amazon Web Services (AWS) provide this service, but AWS is the largest and best known.
Your company rents a set of these from AWS, and moves jobs, databases, and applications from on-site servers in your data center to these AWS servers. Once network connectivity is established, things run just the same as they did on-premise except that you no longer own and operate the hardware. This is infrastructure in the cloud, or IaaS (Infrastructure as a Service). Its a service because you don’t own it, but you rent it “as a service”.
But, the software applications that run on it are yours, or purchased/licensed by you. That part hasn’t changed. You may still be running your licensed version of SAP, but now the hardware it is running on is owned by Amazon, and rented by you. And the application architecture, and integration architecture, is still the same. Once network connectivity is in place, however you were getting data in and out of those applications has not changed.
In contrast, the rise of cloud applications, or SaaS (Software as a Service) has changed the architecture and integration landscape. In SaaS, you are no longer running “your” application, code, or database, but renting a seat who can use the vendor’s fully managed application to execute a specific business process for your company. The biggest and best known SaaS vendor is of course, Salesforce. They started with an application focused on a very common use case - that of managing the sales process and its associated data. This application had two important innovations:
- Multi-tennancy - all data for all users is in the same set of data base tables
- Sold as a service - the customer rents the use of the application per month per user
These two innovations, working together, completely transformed the global $400B enterprise software industry (Gartner, 2107). I’ve written in detail about the hows and whys of this transformation, here and here. The combination of dramatic cost reduction, dramatic feature increase, and removal of buying and implementation barriers made SaaS applications the overwhelming choice for new implementations, and increasingly for replacement implementations of business application software.
Rise of SaaS - Cloud Applications Dominate
In contrast to IaaS, SaaS has changed the architecture and integration landscape, in several important ways. The first way is obvious - you are running your business process on an application and data structure that you don’t own (you just rent it) and you typically can’t access the data structures directly. Thus the rise of the API (application programming interface), which gives you a programmatic interface (structured calls and returns) to your data base. This is different from on-premise applications, where you typically had access to the data structures, and made data base queries to get bulk data that you moved to another data base.
The other impact of SaaS is more subtle - the rise of “Best of Breed” applications, and the fragmenting of the application software space into sub and sub-sub domains. For a number of reasons, including ease of penetrating the market, SaaS applications have focused on a specific business function for a specific market segment. This has lead to an explosion of SaaS applications on the market. No one knows how many, but informed estimates put the number at well over 10,000. No longer do you buy an HR application, you license use of a recruiting and on boarding application for tech companies (such as Jobvite - a bit of googling found a SaaS solution in this randomly chosen segment).
this explosion means that an enterprise no longer has a dozen applications on-premise, but now has fifty applications in the cloud - each one the best of breed for a specific business process (on-boarding, lease accounting, community management, etc). The compelling economics of best-of-breed SaaS applications means that a department can buy their own solution, without central IT support. In fact, some of our fast growing tech customers have no central IT function in the traditional sense, but each functional group sources their own applications.
This proliferation of cloud applications blows away traditional notions of IT architecture and master data management. The cows are out of the barn, IT can just chase them around the pasture. How do you define and manage an overall information architecture when your departments and business units acquire and set up their own applications?
As the application landscape fragments and moves to the cloud, so the business process data created by customers using these applications moves to the cloud as well. This is everything from a job applicant’s address to the pricing of a widget you just sold. This data is now all over the place, in 50 different cloud data bases that you can only access through an API, or programming interface. But, the HR Benefits application needs the addresses from the recruiting application. And the billing application needs the pricing from the sales application. How is all this valuable business data, spread across fifty application platforms, going to move from one to another to enable seamless business processes?
Well, you need integration tools. You need structured means of moving business process data from one cloud application to another.
Enter IPaaS - Integration for Application Fragmentation
Enter IPaaS, or Integration Platform as a Service. About 10 years ago, a very smart guy from Philadelphia, Rick Nucci, saw that all of the above would come to pass, and that there would be a need for integration tools to connect this coming blizzard of cloud applications. So he bet the farm on converting his on-premise EDI software company, Boomi, to the first cloud-focused integration platform, or a segment that would become IPaaS. The Boomi integration was the first, and remains the best set of integration tools for the new “cloud-fragmented” application architecture. Boomi has the important features needed to be a complete solution to the new cloud application architecture, including:
- Huge library of Connectors that talk to the Application APIs
- High performance ATOM runtime engine on premise, or use the Boomi ATOM in the cloud
- Codeless, hyper-productive builder tools that allow you to build and change integrations quickly
- Full SDLC support including build, deploy and monitoring tools
- Expanded product line includes API management, MDM, Queues and more
We will talk more about how this most mature Boomi cloud application tool set fits the needs of the cloud integration specialist.
So, there is a need for enterprises to integrate this new cloud-fragmented application space. But what do these integration architectures look like? In the past there have been a series of integration approaches that moved in and out of favor - including enterprise data models, enterprise data warehouse, SOA, and enterprise service bus. What is the emerging integration architecture in the cloud-fragmented application space? And who would be knowledgable about this emerging IT practice area?
Kitepipe does Integration in the Cloud
Enter Kitepipe, a Cloud Integration Services firm, specializing in building integration processes in the emerging cloud-fragmented application space. Kitepipe’s founder, Larry Cone (your humble author) is another smart Philly guy. I was VP Engineering at an early cloud application company near Philadelphia. So early that we didn’t call it Cloud, or even SaaS, but an ASP, Application Service Provider. We needed an integration tool to connect to our newly-installed Netsuite instance, and I selected Boomi as our integration tool. I knew about Boomi and Rick through the local IT ecosystem, and they were a natural choice. I took the training, and did some hands-on building with the Boomi tools, and I really liked the product.
My insight was a bit later. I too saw the coming fragmentation of the application space into the cloud, and wrote about it here and here. I too saw the coming need for integration between these cloud based applications. I too saw that Boomi was a great toolset, purpose-built for this role. I started a cloud integration services company, Kitepipe, to address this coming market. Kitepipe is today a leader in implementing and managing Boomi integration suites for companies large and small.
One Additional Insight
And I had one additional insight, after working at integration for a while, which is this: That the value to the business comes from thinking about integration at the business process level, rather than the data level. So, don’t think about moving closed-won opportunities from Salesforce to Netsuite - think about how the integration process can complete the sales-fulfillment-billing cycle across multiple applications. Understand the operational challenges, and build an integration process that solves those challenges.
But what does the integration architecture look like? At last, we are getting to the core of this series of posts. Lets take that sales process as an example, deconstruct it, and look at the architecture that results from implementing using best practices.
Often, the best way to proceed is to understand the set of problems to be solved. I call this "selling to pain” - its the approach I use in the sales process to convey the value of the proposed integration project. so, here are some typical pain points in the sales to accounting integration:
- different field and data structures
- different code values for the same data
- wrong customer
- wrong/bad pricing
- special or promotional pricing
- wrong territory or geography
- wrong currency
- related deal components - not SOX compliant
- all happens at the end of the quarter
- not well forecasted
- new products/offerings not yet in accounting
The number one role of the integration is to transform fields and codes from sales format to accounting format, but that is just the start. Most of the above issues can be characterized as “bad data” - data from Sales that does not fit (yet) into Accounting. An effective integration approach needs to start with the understanding that most integration problems are data problems. Based on this, we at Kitepipe have developed a template for building highly functional integrations.
Functions of a Rich Integration Process
So the number one role of the integration access is access - the process needs to be able to selectively query the data in the right state; the next role is Transform - map source data to target data profiles; The number three role is Enrich - look up or add master data to the transaction, as needed; the next role is Validation - is all the date good? the fifth role is alerting - tell someone who can fix the data. The sixth role is to Post the transaction to the target application, and detect any errors; The seventh role is to Synchronize - make sure that there is a key pair between the application platforms that can be used to match the transactions for future updates. The next role is Recovery - the integration process should be recoverable in case of failure, ideally by just running it again. The last role is Status - make sure users on either side know what happened, and what the status of the transaction(s) are.
So thats:
- Access - select the data you want
- Transformation - map source to target
- Enrich - include data from other sources to enrich the transaction
- Validation - insure that the data is valid
- Alerts - notify those who can fix the bad data
- Post - post the new or updated transaction to the target system(s), and detect any errors
- Synchronization - post keys to target, and back to the source to confirm traceability
- Recovery - structure the process for recovery, ideally by rerunning, and without intervention
- Status - post status to logs, people, source and target platforms
This set of features is built in to our process templates at Kitepipe, and speeds the integration development process. Because Boomi is a powerful purpose-built platform for building integrations, all of the above functionality can be quickly built, tested and deployed, without writing code.
These functions of the integration are architecture at the process level - a Boomi process can (and should) accomplish all of these functions for a given set of transactions. There are additional choices to make which impact how the processes executed - batch vs. real time, for instance - but in Boomi the integration structure and logic is the same, the only difference is how the process is invoked. You can select a batch of records from the sales system and process them, or deploy a “listener” or web service - that accepts calls from a trigger in the Sales system to process sales records on an event basis. But in Boomi the process structure is the same.
This represents a ‘Point to Point” integration solution - from the sales application to the accounting application. Many of the integrations we build at Kitepipe - and we build a lot of integrations - are of this type. - but surely this is not an enterprise integration architecture? Well, yes it is - a point to point integration solves a business process problem quickly and effectively.
But what about my complex enterprise where we have possibly fifty application endpoints, and and the possible point to point integrations is in the hundreds - in fact, Siri tells me that there are 1,250 unique pairs of integration points of 50 applications - twice that if direction is counted separately. This rapidly becomes unmanageable, and a different approach is needed. Or is it…
Having looked at long lists of application platforms in a single enterprise, I can confirm that the vast majority of application pairs, taken randomly, don’t make sense, and would not be implemented. So, you don’t need to manage 1200 integrations. Our most complex customers have implemented 30 or 40 integrations, which is manageable by a small, well-trained integration team. But not all of those are point to point - we use some integration hub techniques to manage complexity in complex enterprises.
Evolution of Cloud Integration Architecture
Probably the best way to think of integration architecture is based on evolution - what do you do first, and next, and what might the final state look like? This is a good approach to take, because it allows quick wins on the way to a more sustainable architecture.
Ah for the (bad) old days, when a corporate IT shop could embark on 24 or 36 month or 5 year re-architecture or master data implementation projects, which got repeatedly delayed, then quietly cancelled. No one has that kind of time or money anymore. We at Kitepipe need to sing for our supper every night, so we better be putting something into production that improves the client's business every six to eight weeks.
So, how do you implement a complex cloud-fragmented application integration architecture? As the old riddle asks, how do you eat the Elephant? One spoonful at a time.
Here is our successful recipe for building a comprehensive cloud integration architecture for the enterprise.
↧
↧
Kitepipe recognized as leader in Netsuite integration space by CIO Review
In the latest CIO Review, focused on Netsuite solution providers, Kitepipe was recognized as a leader in creating high value integrations using Boomi.
↧
New Video - Why is Kitepipe Unique?
As a Select Services Partner, Kitepipe has a trusted partnership with Dell Boomi. In this new video, Kitepipe CEO Larry Cone talks about:
- Why he began a company that only does Boomi integrations,
- What he feels is so successful about the Boomi platform., and
- How Kitepipe's focus had lead to our success
He dives into the business benefits of a Boomi integrated system and why Boomi will take over the (application) world.
↧
Kitepipe a Diamond Sponsor of Boomi World 2018!
I am pleased to announce that Kitepipe is a Diamond Sponsor of the second Boomi World conference. The Conference is being held at the Wynn Las Vegas on November 5th -7th.
↧