A vitreous SkyForm cloud platform architecture analysis
In this article, a vitreous Jia Kun software r&d director from the open source IaaS technology trends and the actual demand of the private cloud, according to the product research and development of the vitreous and project practical experience are introduced SkyForm cloud management platform architecture and implementation approach, involved in asset management, deployment, monitoring, operational and service management, etc.
The actual demand of IaaS private cloud project
Last year it Meetup in offline activities, someone summed up the domestic private cloud project based on it to meet the demand of the following functions:
1. Based on it to provide the service function of the infrastructure
layer, including virtual machines, storage and network function,
2. Need complete monitoring functions, including hardware equipment,
the Hypervisor physical machines and virtual machine or application
3. Considering the demand of ITIL, such as providing asset management,
CMDB, user/permission management, audit and event notification,
and other functions;
4. Consider the resources integration and the existing system
migration, such as VMware virtualization stock of integration
Similar views we can also from Forrester survey released last year in 10 facts about private cloud: three core components of private cloud is the service access, tracking and monitoring resources and fully automated;The most time consuming step;Integration and process automation.
Comprehensive two points, we get the following typical private cloud IaaS plan:
; Private cloud self-service access through it or CloudStack and
other open source IaaS platform provide;
; Track resources functions require the system to provide the
resources such as asset management, CMDB from procurement to online
business and various changes in the middle of the whole life cycle
; Collecting and alarm monitoring resources cover configuration
management, performance features;
; By monitoring and warning system of fault, event handling and
related to maintain the stability of the system to work effectively,
build a closed loop feedback system to build fully automated
Traditional enterprise IT management popular ITIL through a set of norms to guide IT service management, under the development of the cloud is the realization of higher requirements, to build a tool chain to achieve full automation, this is the cloud management platform.In the process of the goal to achieve, all kinds of system integration and process automation is the current norm, is also a starting point.Plus cloud bursting requirements, basic covers the technology development direction of the private cloud project and the project be born on the system architecture and function requirements.
A vitreous SkyForm private cloud management platform for practice
A vitreous software product development and project practice in recent years gradually built a cloud management platform products SkyForm to try to meet the above requirements, the following introduce the function of the system, architecture and realization of ideas.
SkyForm system architecture
From the system architecture diagram (pictured above), we can see in SkyForm provides unified, IaaS self-service function abstraction, can cover all kinds of public cloud, private cloud system;Also design the function of resource management, user rights management and monitoring, alarm function.
SkyForm Service is at the heart of the child from SkyForm Cloud Service Gateway, SkyForm Cloud API, and a variety of functional components, Cloud platform driven SkyForm composed of Multi - Cloud Engine.Including SkyForm Cloud Service Gateway to all sorts of private Cloud and public Cloud undertake unity of the abstract and manage various system ability of different SkyForm Cloud API is providing abstract it compatible API and provide a specific function extension.Virtualization platform (such as VMware), open source
cloud OS platforms (such as CloudStack and it), container technology (such as Docker) as well as the public cloud (AWS) through the Driver access SkyForm cloud platform to meet the needs of a variety of scenarios.The VMware Driver will handle user VM stock of automatic discovery and self-service management operation, can use VMware advanced resource management functions such as DRS and support a variety of integrated VMware IaaS services such as VDP backup service, desktop services View, PaaS layer such as vFabric database services.
SkyForm Resource Scheduler provides Resource management, Resource scheduling, Resource analysis, and other functions.It deals with the resource distribution and location decision-making of the VM, public Cloud Cloud Bursting trigger and provides a variety of priority-based/resource management ability, the capability of implementation by implementing centralized it/cloudstack scheduling plug-ins.Scheduler resource capacity also provides system management functions, plus resource request analysis, request prediction, trying to provide enterprises with cloud platform construction investment decision-making support.
SkyForm Application Manager to manage the cloud run business used by all kinds of IT resources, including the VM/physical machine, the virtual hard disk and network resources (such as VNetWork, VLAN, IP) resources statistics, detailed list of information resources and resource usage information (such as a VM as a DB server and access server USES information).Cooperate with Monitoring as a Service Service and application performance data collection technology, can provide complete business resource management and application performance management (APM) solution.
SkyForm CMDB design meet the needs of asset management, CMDB and ITIL, IT connected to the RabbitMQ message bus, listening from SkyForm Multi - Cloud Engine and other system resources life cycle change events (such as the creation of the VM/start/stop/destruction and the Hypervisor downtime events), maintaining the overall relationship in IT resources and change information.IT connects the SkyForm OS (Multi - Cloud Engine) and management systems (monitoring and alarm systems), as well as API integration with other IT systems.
SkyForm designed a distributed data acquisition, analysis, statistical framework, to provide from the physical machine Hypervisor, switches, a storage device to the virtual machine monitor, alarm support.From above, include centralized SkyForm sampling control node and the task execution, this design can effectively handle huge amounts of data acquisition task mass environment, also can deal with complicated data acquisition under the network environment and safety requirements.Collect data directly perform various processing, threshold alarm rules will alarm unified summary of various kinds of equipment including resource instances.Alarm system supports external equipment or system through SNMPTrap or message active abnormal report all sorts of equipment, early warning system.To take advantage of excellent open source to collect the ability of monitoring and control system, designs the collection adapters to support the Zabbix expand SkyForm equipment support ability, etc.Based on it the other Celiometer developed Monitoring Service (Monitoring as a Service), used for user-defined resources index and application performance management.
SkyForm overall architecture and implementation approach
SkyForm are analyzed in detail in the design of various open source cloud platform technology (see related article"Analysis and comparison of open source cloud IaaS platform"And absorbing the team members involved in the lessons and experiences for similar products, the overall architecture and implementation has the following several aspects:
1. Layered with SOA design
Clearly define the Portal system with API access layer, the core function of IaaS layer, interface adapter layer and the underlying IaaS resources such as logical layering, has the function of independent services, such as Multi - Cloud IaaS Service Engine, the Scheduler SkyForm Resource Scheduler, Monitoring Service Monitoring, Alarm Service Alarm Service, including the use of did not appear in the architecture diagram AutoScaling services and to address the Service discovery and rely on Service management capabilities.
2. Asynchronous event-driven
Asynchronous distributed system is the typical way of information exchange, SkyForm introduced the RabbitMQ to take advantage of the asynchronous events.For example, the Multi - Cloud Engine receives the underlying Cloud Service Gateway from the various Cloud resources change events, such as the creation of a VM start/stop, the destruction of such events, at the same time foreign radio unified resource changes after transformation event driven SkyForm CMDB and monitoring CollectorManager, the Collector to make corresponding processing logic.In the system design of the global event management function is used to record all kinds of system events (such as user login, VM operation, the system
configuration, etc), can be as a resource metering, security audit, process tracking, and other functions of the original information.
3. The control flow and data flow separation
Cloud platform is actually a management system, including resource opening process, several important processes such as resource operation process.The message bus design can effectively deal with the process.On the other hand usually manage resources between different service is a dimension of information, need to maintain key information synchronization between the service also provides data query capabilities.SkyForm principle is the framework of a control flow driven by an asynchronous message, service data exchange between the Restful API synchronization is complete and supplemented by asynchronous event messages for incremental updates.
4. The distributed task management
SkyForm's two most important service components, Multi - Cloud Engine and monitor service adopts the distributed design.The Multi - Cloud Engine USES the load balance design of statelessness, monitoring service, the use of the Master - Slave master-slave design collection processing task management and load balancing mode response to external query.An important starting point is based on the resource lifecycle from a large number of users more concurrent operation without dependence on each other, and there are all kinds of cluster monitoring acquisition task, regional, global scheduling of network security restrictions need to be flexible in order to meet the requirements of the production environment.Future monitoring system refers to Gossip and distributed hash technology evolution of architecture and complementary health care system, meet the very large scale of unattended cluster and task management needs.
5. The plug-in design
Plug-in design is the classic practice to realize heterogeneous systems management, SkyForm in Multi - Cloud Engine, performance data collection, external alarm accept processing of similar design has been adopted to deal with heterogeneous multi-version IaaS resource pool, multi-type acquisition task and diverse source of alarm management requirements.
The next step of work
Typical IT system construction, practical, usually need to experience maintenance, tuning and reinvestment stage, cloud management platform as a new generation of enterprise IT resources supply technology platform, also need to provide the technical support from the Angle of the IT service life cycle.SkyForm or similar solution based on CloudStack/it provides users with IaaS layer resources building clouds and service technology platform, but the user needs according to their own business needs to build a suitable cover application development, testing, online management of application lifecycle management platform;Also
once the cloud platform system running, service monitoring, system becomes daily work first, the user urgently need a high automation level of operational technology platform.Aimed at the fact that SkyForm current and future will be on the application of management and operational technology to do some exploration, in this process will introduce technology of big data from resource life cycle dimension to collect relevant data, through the statistical analysis to tuning for resources, resource scheduling, and provide support to investment decisions, etc.