Reference Architecture Introduction

In today’s age of on-demand access to applications, compute, storage, and networks, modern IT applications and service management has many complications:

• Applications can be deployed across organizationally & geographically distributed data centers. The technology in these data centers, from virtualization platforms, to host, storage, and network infrastructure, is typically heterogeneous, and not necessarily managed with uniform policies and interfaces.
  
• The performance, scalability, and availability characteristics of an application are due to a complex combination of design and operational decisions. The greatest impacts on these factors are due to decisions in the architecture and development of the application, before configuring the data center infrastructure.
  
• Application and infrastructure management is complex and inter-disciplinary. It’s unlikely a system can be diagnosed and maintained by one person to keeping the system design & configuration in their head. Application design, administration, and management typically is a collaborative activity across specialists; there is no “one-size fits all” design tool, management tool or application platform.

Enterprise IT management challenges have been tackled through a variety of modeling languages, models, protocols, APIs, formats, and methodologies to date, each of which tackles a portion of the problem.   But to date there has not been an end-to-end approach for describing an application, from design, through configuration, and operations. It is our opinion that an end-to-end, model-driven approach to IT service design, management, and automation, provides great potential for drastically reduced lead times to create and maintain the next generation of “cloud applications”.

Three design goals for an end-to-end cloud design approach include:

Separated Applications from Infrastructure, through modeling the application in terms of its architecture and infrastructure requirements, without tying the application to a specific set of underlying infrastructure

Enabling Computer-Assisted Modeling and Control Automation, provided by a set of control agents and user-guided by graphical design tools. This could help IT architects and operators determine design constraints on the application, match the design to the underlying infrastructure, and enable goal-driven automation to deploy, scale, or recover their IT systems on demand.

Explicit Collaboration To Enact Changes, through models that codify, relate and analyze the constraints and preferences that are appropriate to stakeholders across enterprise IT: from architects and developers, through operators, administrators, and managers.

Characterizing an Integrated Design & Management Approach for Cloud Computing

We have defined a set of modeling languages, a reference architecture, and built an implementation, which integrates both existing and emerging IT automation and management servers. This work is based on a set of eight desirable characteristics that describe an information system that addresses cloud application design and operations problems holistically.

The eight characteristics are:

1. Distributed, Autonomous Control

A cloud design and management system must be able to scale, remain reliable, and interoperable across an unpredictable number of organizations and/or locations. This recognizes that infrastructure resources, design information, and deployments are normally decentralized.

2. Open Document-Exchange

Interactions among users, agents, and management services must be visible and persistent if they are to be managed. It’s common, for example, that administrators wish to explore “what-if” scenarios that express the desired state of an IT system, comparing to its current state, before enacting a change. API-based protocols, while open, do not directly support such abilities.

In practice today, persistent, visible interactions are usually managed via proprietary, centralized or federated configuration management databases (CMDBs). While partly effective, the CMDB limits a number of the desirable characteristics listed here. CMDBs are oriented towards centralized, or at best, federated data management, instead of distributed data management.  And the CMDB schema limits visibility due to its typical proprietary nature, although query standards are emerging.

To contrast, the exchange of documents, whether via network transfer, portable storage device, or email attachment, enables freedom for users to separate the mode of interaction with the content of the interaction. For example, a document may describe a scale-out plan for a deployment.  In a document-centric system, the document isn’t wedded to any particular intent, and can be used for a variety of purposes: collaboration among colleagues, archive for search and analysis, or for actually performing an automated scale-out.

3. Hyperlinked Web Architecture

The architecture of the World Wide Web, based on the REST architectural style, should be the default network architecture for exchanging documents among services, agents, and users.

A worrisome trend among document formats is the desire to shove every possible item of data and metadata into a single monolithic entity. This leads to monolithic specifications, and incompatibility among different IT management technologies. Practically speaking, large “deployment descriptors”, “manifests”, etc., are hard to interpret, debug, and share or collaborate with.

Hyperlinks extend the potential of the document exchange; it turns a document into both an item of information, and an active node on a network. A document can link to other real-world resources, people, and other documents. It enables traceability between different document formats and technologies, such as the Open Services for Lifecycle Collaboration (OSLC), a set of specifications for describing application lifecycle models, or the Distributed Management Task Force’s OVF, a standardized deployment descriptor and package for virtual machine images. Ultimately, the most complex IT system description can be broken down into a web of continually evolving, interconnected documents, which can be securely searched, indexed, and processed.

4. Model-Driven

Application management, automation, and control actions must occur within the context of many interrelated technologies and design decisions. Typically these assumptions are encoded in software such as management system plug-ins, scripts, or runbook processes, with limited ability to compose or extend the results to contexts beyond why the software was originally created.

Models, on the other hand, are expressed as data, not (just) code; are descriptive rather than proscriptive; and can be interpreted in different contexts.   Models are easier to change than code, are easier to compose than executables, and can be presented both visually (as a visual modeling language) and textually (as, for example, a domain-specific language).

Models also have the benefit of being able to take advantage of logic and semantic reasoning to increase the visibility and interoperability of the information conveyed. Semantic Web technology such as SPARQL, RDF and OWL brings logical query, representation, and modeling to the Web Architecture.

5. Goal and Policy-Driven

An IT management system ultimately provides information and support to architects and administrators to make better decisions. A goal-driven approach to IT management, described, for example, by IBM’s Autonomic Computing vision, separates the desired state of the system from the potential means of achieving that state. A policy-driven approach, meanwhile, expresses certain preferences or constraints on the system as it is transitioned through its lifecycle, without requiring administrators to specify the complete maintenance process for every contingency. Both designers and administrators can inspect, diagnose, and modify the automation processes through automated plan generation.

Traditional automation based around scripts or runbook processes executes a well-trodden, predictable path, but the technology doesn’t help improve the processes themselves. To contrast, declarative programming, such as with goal and policy-driven models, enables machine reasoning and deliberation on behalf of system administrators.   Axioms and procedural descriptions can be captured into operational knowledge modules that contain the various states, events, conditions, composite tasks, and actions in a particular domain.   This computer-assisted approach to design and operations can assist administrators and architects optimal guidance for certain design decisions or for a plan to enact changes. This goal and policy-driven approach is implementable through an automated planner, which can interpret the goal, model, and policy in the context of a set of operational knowledge modules which describe the planning domain.

6. Viewpoint-Based

Application and infrastructure architectures typically involve multiple stakeholders, each with their own unique view on the important concerns and notable elements of the system. The systems architecture and software engineering communities have coined the term “viewpoint” to describe a reusable language, set of constraints, and practical guidance to describe an end-to-end system that address a particular set of concerns. For example, a security viewpoint may focus on vectors of attack and policy enforcement, decision, and administration points in the architecture, where a performance viewpoint may view the system as a queuing model to analyze end-to-end response times.

A viewpoint-based approach to modeling architecture provides the foundation for the intelligent automation configuration of administrative goals across the declared operational knowledge in the architecture. Different aspects of the system can be modeled in the terminology most familiar to the domain expert, with conflict and precedence evaluated by the control plane, in part by transforming the various viewpoints into a foundation model, such as a component and connector viewpoint.

7. Collaborative

As the scale of an IT application or system grows, collaboration is essential, as multiple people will be maintaining different parts of the system simultaneously. Communicating and sharing models, change plans, and historical effects can reduce configuration conflicts and operator errors.

Hyperlinked models can take advantage of the web architecture’s rich support for social computing and collective intelligence, including aggregation, syndication via feeds, and the inherent ability to break down a difficult problem into discrete, distributed hyperlinked chunks of information that can be shared and edited.

8.  Governable

The increased freedom that is growing in automated, virtualized IT environments carries a cost: the increased amount of virtual infrastructure that will be created, and needs to be managed, and whose costs and risks need to be governed.

Our reference architecture specifies natural entry points for control agents to evaluate and enforce of IT governance policies.  Governance may span the lifecycle of and access control to features of an application, security policy configuration, resource budgeting, and license compliance reporting.