EM7 G3 Technology Overview

Photo of EM7 Meta-Appliance Cloud-Ready Network Management Fabric for globally distributed network operations.
technical diagram of EM7 G3

View full-size image

The diagram above provides an overview of data flow and communciations for a sample high availability EM7 G3 solution.

Technical Highlights:

  • Robust monitoring solution designed to handle large scale - up to 10,000 devices or 3.6 trillion database queries daily, all utilizing a single data store for data synchronization that supports faster troubleshooting
  • Decentralized architecture for maximum processing capability, increases efficiency of monitoring operations and ensures integrity of centralized data repository
  • Redundancy and automated failover built into data storage, data collection and user interface levels for true High Availability and Disaster Recovery solution

Larger and more complex networks require huge volumes of granular information to be collected, analyzed, stored and reported. To meet the challenge, EM7 G3 is based on an entirely new decentralized monitoring architecture that creates efficiencies and scale by moving processing out to the edge. Collector Resource Groups (CRG) automatically load balance data collection in the event of individual collector failures, using a unique Store and Retrieve technology that ensures that no performance or event data are missed for mission-critical operations.

EM7 G3 includes a Central Data Repository (CDR) – which is capable of managing up to 10,000 managed devices and stores all configuration, statistical and business data. The CDR is responsible for events, alerts, notifications and scheduling of jobs – including task allocation and data retrieval from CRGs. Configured in a cluster mode where one database server is operational and the other runs as a standby, the CDR utilizes a SAN for a true high availability solution with built-in automated failover.

Data compression technologies enable EM7 G3 to use about 65% less bandwidth than typical monitoring solutions. The decentralized monitoring architecture utilizes NSA AES-128 encryption and eliminates security risks found in traditional centralized monitoring solutions by locking down critical system access information that is stored and protected locally at the edge – particularly important as monitoring is extended outside the enterprise to public clouds.

Decentralized Extensible Architecture

Similar to EM7 G2, the core functions of EM7 G3 are distributed to three "tiers" of dedicated systems:

  1. Data Collectors
  2. Centralized Database
  3. Administrative Portal

The three tiers can be geographically dispersed depending on customer specific IT Infrastructure location and security requirements. Distributing functions in tiers enables customers to maximize the performance of the appliances that make up the total EM7 G3 solution. Each dedicated EM7 Appliance has been tuned for optimal performance of specific EM7 functionality. For example, each Data Collector can manage hundreds of nodes and independently collect and correlate syslog and other forms of inbound messaging. Data Collectors can be positioned almost anywhere in your network, giving additional fail-over and fault tolerance for remotely managed systems and devices.

EM7 G3 Decentralized Architecture vs Traditional Centralized Systems

EM7 G3 utilizes a centralized data repository for a synchronized data store of fault, performance, configuration and ticketing information - up to 3.6 trillion database queries daily. In order to handle such scale, it was necessary to create an entirely new monitoring model that works on a decentralized basis - moving data processing out to the edge where devices resides and utilizing advanced access and security features to ensure the integrity of the centralized data repository.

Traditional centralized management systems require administrators to open a vast number of ports to allow traffic to flow back for anything that needs to be monitored. In our decentralized system, traffic from each monitored device only needs to be able to reach the Data Collection Appliance instead of going all the way back to the central monitoring system which could be behind firewalls or have additional security restrictions.

This architecture not only reduces the number of open ports and firewall access for monitoring traffic, but it also reduces the amount of data that must be carried across the WAN to that required by just the retention and event rules. Log parsing, trap handling and polling can all occur at the Data Collection tier. In other words, heavy communication between boxes and Data Collection Appliances(s) can happen locally on the LAN where bandwidth is plentiful instead of across the WAN where bandwidth may be limited, expensive or even unreliable.

Hot Standby for High Availability and Disaster Recovery
EM7 G3 includes extensive failover and redundancy features for true High Availability monitoring. EM7 G3 utilizes three primary forms of High Availability services: Clustering, Data snap shot, and Collector Grouping. Clusters are created by deployment of a SAN. Redundant EM7 G3 database appliances associated to the SAN monitor each other and continue operations in event of a failure. Data snap shots or archiving can also be performed without interruption of operations. These archives can be stored for disaster recovery and used for rapid restoration after a catastrophic event.

Collector Resource Pools for Load Balancing of Data Collection
Unique to EM7 G3, collectors residing on different network segments may be "grouped" to form Collector Resource Pools that perform automatic load balancing and failover for data collection. In the event of a collector failure, other collectors in the same group will be tasked for the required work.

Collection of Data from Devices and Integration with 3rd Party Solutions
EM7 G3 utilizes a combination of both passive and active methods for gathering data and event information from devices and existing third-party solutions - regardless of vendor or operating system.

Passive Monitoring: EM7 G3 can accept email, traps and syslogs, Informs
Active Monitoring: EM7 G3 polls IT infrastructure using:

  • SNMP v1, v2, v3
  • HTTP/S to retrieve XML-formatted data or to perform Web Content Verification
  • SOAP/XML
  • SQL queries through the Dynamic Application engine
  • SMTP for round-trip mail performance testing
  • ICMP checks
  • TCP Port checking

Web-Based Access, Notifications and Desktop
Users access EM7 via HTTP/S in a Web-based console that can be customized based upon access level, function, geography, etc to show the devices, events, tickets and other EM7-collected information needed to help them perform their jobs. No client software is required. EM7 sends customized notifications for events and tickets via email and RSS feeds to desktops or handheld devices.

EM7 G3 also features a desktop application developed in Adobe AIR that operates as a quick overview of status and events and serves as a launch pad for detailed functions.

Agentless and Agent-based Monitoring
Most of the functionality provided by EM7 G3, particularly that based upon SNMP monitoring, does not require agents to be installed. Some systems may have pre-installed agents, e.g., Dell OpenManage, IBM Director and HP/Compaq Insight Manager, which are used by EM7 Dynamic Applications to provide very granular monitoring. Customers may require WMI agents for advanced monitoring of MS SQL, MS Exchange, MS IIS and automated server stops and restarts. EM7 G3 also supports SOAP transactions at the API level for access to systems such as Cisco AXL, NetApp Z-API, or VMWare API interfaces.