Chapter 2:
Streamlining global operations
MagnaMotors PLC is a British automotive company headquartered in London. It is a key strategic partner to the world’s foremost premium and luxury car brands, providing a customer-centric route to market for vehicles and parts.
Infrastructure
A simplified view of MagnaMotors' IT infrastructure
Services are split into global and regional services. Global services are handled and requested across all regions while the regional ones are limited to the specific region or country.
20k Users
250 Technicians
30 Sites
A quick look at MagnaMotors' user stats
The global service catalog has a manager who then assigns regional heads for each process. This means there's a manager for critical incidents, one for change management, and so on. Local requests are approved by the regional service managers. These requests are typically singular in nature, like a new asset request. Global requests, on the other hand, tend to affect operations across all branches. For instance, adding a new service request category requires validation from the local team and is then pushed to the global team for process completion.
The challenges
MagnaMotors' infrastructure complexity stems from its widespread market. It chose ManageEngine primarily due to the multi-site support we offer, which helps manage its users worldwide with custom workflows and configurations. However, the uniqueness of its multi-site requirements posed a few obstacles, like:
1. Lack of uniformity
MagnaMotors opted for multiple help desk systems across different regions with separate licenses. This created confusion and provided limited understanding of the overall system for decision-makers. Management found tracking global trends and performance to be a herculean task.
2. Rigidity in services
With segregated instances, MagnaMotors was unable to move tickets across countries and escalate tickets to global support when needed. It needed to merge these services into a single global instance.
3. Poor visibility
Once a ticket was created and assigned to the global services team, it was no longer visible to the regional team. This defeated the purpose of localizing services for users.
4. Complicated workflows
When a ticket was assigned incorrectly, technicians were unable to revert changes. They had to contact the IT team to correct the ticket status, making it a cumbersome process.
Global process catalog
1. User access management
MagnaMotors owns multiple manufacturing plants worldwide, and the user access management module enables it to streamline services. The workflow below is primarily for any access request raised by users across different services and within any region.
Each factory floor has multiple lines, each tasked with its own responsibilities and handled by a specific group. Let's say the line that handles motor assembly requires access to a remote panel. A member of the floor staff (user) can raise a request for a service and have it approved by the line manager. After approval, the regional IT admin (IT service owner) evaluates the request feasibility. Since floor staff are usually employed on a contract basis, the IT admin can go ahead and provide temporary access. The entire life cycle for user access can be monitored by regional and global IT teams on this module.
2. Critical incident management
Our implementation team set up a critical incident management process for major issues. Critical incidents typically include scenarios like malfunctioning machinery, factory shutdown, parts shortage, or vehicle breakdown. For a company like MagnaMotors that generates billions in annual revenue, every stage of its incident management process has to be mapped out to the last detail.
What do you need for an effective critical incident management plan?
a. Critical incident team
A team consisting technicians, service managers, and organization stakeholders should be assembled and notified in case of emergencies.
b. Critical incident management process
A detailed workflow with clearly assigned roles and responsibilities should be documented in the organization knowledge base.
c. Incident management tool
Managers, stakeholders, and key decision-makers should be able to oversee the entire incident management process. Additionally, they must be able to provide feedback, reviews, and provide ad hoc approvals.
Critical incident management process
The process is shaped by four core methodologies:
1. Identify
- Communicate with stakeholders regarding incident process.
- Activate a critical incident process.
- Conduct initial incident review.
2. Assemble
- Set up a critical incident team.
- Provide critical incident review.
3. Resolve
- Provide technical details regarding the incident to the technical teams.
- Present real-time updates to the stakeholders on the incident in progress.
- Collaborate with technical teams, leads, and stakeholders to identify a suitable solution.
4. Maintain
- Conduct root cause analysis (RCA) of the incident and present findings to the critical incident team.
- Determine ways to improve actions based on the RCA.
- Set up periodic meetings with service owners to review improvement actions and avoid new occurrences.
Core methodologies for critical incident management
3. Global knowledge management
The aim of a knowledge management module is to create a centralized knowledge base accessible to MagnaMotors users and technicians worldwide.
Upon entering the category information and symptoms of the issue, the knowledge base provides suitable suggestions. Let's take a closer look at why this works for MagnaMotors' IT team. A technician at the branch in Brazil is facing an issue with a local OEM partner. They are unable to update certification details for new vehicle parts. The technician enters the required details along with symptom "new parts option unavailable." Since MagnaMotors has OEM partners worldwide, this isn't the first time this issue has occurred. Thankfully, the knowledge base contains a detailed guide that can assist the technician.
In addition to a detailed resolution, the article also contains the following:
Article ID |
AB1234-56 |
Title |
New parts option |
Published date |
27 Jan. 2022 |
Review date |
28 Jan. 2022 |
Expiry date |
28 Jan. 2024 |
Internal comments |
Verified by global IT |
Created by |
John Doe [Singapore] |
Last updated by |
Jane Doe [UK] |
Visibility |
All L-1 technicians |
Status |
Resolved |
Keywords |
New parts, certification, vehicle parts |
A unified knowledge base allows collaboration between technicians across the globe, simplifying tasks and reducing their ticket resolution time. For OEM partners, this enables faster movement across the supply chain.
4. Global change management
The change management system set up within MagnaMotors' help desk became the centralized change governance across entities, regions, processes, and operational functions.
Change initiation workflow implemented by MagnaMotors
The figure below helps us gain a better understanding of the relationship between change management and other processes in the IT environment. Processes including incident resolution, service requests that cause environmental changes, demand management, and continuous improvements are all be handled via the change management module in the help desk.
Change management's interaction with other processes
Consider a scenario where a user is experiencing performance issues with the order management application. They are unable to add new orders under their region. The user quickly raises an incident ticket on the help desk. The admin's role here is to conduct an RCA to identify the root cause. Upon investigation, they find that the recently patched application is writing erroneous logs to the file server due to a bug.
Now, the correct approach would be to revert the patch. However, the issue was discovered only after the rollback window. Since this is a business-critical function, MagnaMotors cannot afford to take the application down and resolve the issue.
Change management process
The admin decides to initiate a change. The temporary solution is to add space to the file server. By expanding the storage capacity, the application's performance can be managed until the corresponding team decides on a permanent fix. This change goes through a Technical Advisory Board (TAB) and, if required, a Customer Advisory Board (CAB) before it is tested, implemented, and reviewed.
Endpoint management
MagnaMotors operated like an MSP, headquartered in London but having offices all over the world with their own IT team and budget. Each region was allotted a budget and given the choice of their own tool. There were multiple offices under each region but all the offices had limited interaction between technicians. Between regions, communication was restricted to the heads of the regions; technicians were not involved.
MagnaMotors' IT environment consisted of 250 global technicians,
15 global administrators, and 190 regional administrators. Administrators handled digital services for workstations, which involved tasks like
patching, software deployment, remote control, and scripting. Technicians handled assets.
Why did MagnaMotors switch to ManageEngine's endpoint management solution?
1. Compliance
MagnaMotors' main goal was NIST compliance. One important criteria for achieving compliance is having a complete inventory of all assets in an organization, something it did not have due to its decentralized architecture. It didn't have consolidated data earlier.
2. Cost
Spending money on various tools to do the same job left MagnaMotors with little to no wiggle room in its expenses. Once MagnaMotors realized it overshot its IT budget (consistently), it opted to pick one solution. Now, it has a central budget for the IT team that all the regions work with. Even with ManageEngine's enterprise edition, MagnaMotors is well within its IT budget.
3. Security and visibility
With a single solution, it's easier to spot vulnerabilities across systems. Prior to the move, remote or hybrid users didn't have immediate solutions to their problems. Machines could only be patched if the device was within the office network. However, with an endpoint management tool, they can get it done even if they aren't within the premises. MagnaMotors now has total coverage, which made a huge difference during the pandemic.
4. Maintenance
The technicians can reduce the human workload since many tasks like patching can be automated, which was done previously conducted ad hoc.
Implementation
MagnaMotors decided to centralize everything in its network using our endpoint management solution. MagnaMotors set up a main server in the UK, onboarded agents in every country, moved that data to the help desk, and set up a CMDB.
The organization had over 20,000 endpoints in 30 regions. It has a local IT team with technicians who can access assets under their region. For global operations, MagnaMotors set up a centralized service delivery operation team. It has a global dashboard with endpoint count and other details.
ManageEngine's team assisted the global team with setting up its endpoint management system from scratch. We tested it out with four endpoints and scaled up across all 25 regions within a year.
Challenges during implementation:
1. Distributed server (DS) reachability
In MagnaMotors' architecture, it has a distribution server and endpoints. If a device is within the office network, it takes patch updates from the local distribution server. Remote devices, on the other hand, use a central distribution server. The difference in location of these devices is determined by their IP address, which is usually defined by a range.
In MagnaMotors' situation, however, there was a clash in regions. The same IP address, let's say the 192. series, might have two offices. This made segmentation a confusing task and posed a security threat. To combat this, we switched to verifying DS reachability using agent fallback. The agent checks if the DS is reachable (if it's in the internal network) and if not possible, then moves to the central server to take the patch. This way, it no longer relies on the IP address.
2. Custom groups
MagnaMotors had multiple technicians working on the same issue, often in different shifts. Communication was not streamlined, access was not evenly distributed, updates and comments couldn't be viewed in one place, and access couldn't be provided to all the technicians involved. We created a custom group option to resolve these issues.
End result
If you've followed our journey as an enterprise, you know customer input defines the trajectory of our product growth. We listen to our customer requirements and build features accordingly. Some of MagnaMotors' requirements helped us come up with key capabilities for endpoint and IT operations. We were also been able to share our workflows and processes to help establish a fortified endpoint management system for MagnaMotors. Here's how MagnaMotors has enhanced its environment:
1. Controlled bandwidth
Operating in regions with low bandwidth proved to be a challenge. Since each location has its bandwidth variation, we created an option to set limits for each agent to operate, e.g.: 1MB for file downloads.
2. Multi-site connection
MagnaMotors was operating without site-to-site connection, which was inconvenient for tasks like resource access between branches. Site-to-site connection has since been set up with the help of our team.
3. Automated patch updates
Patch updates were carried out ad hoc, leaving them in a tricky position security wise. After implementing our solution, MagnaMotors was able to run on autopilot and keep tabs on patch updates.
4. Simplified operations
MagnaMotors was able to narrow down their operations by eliminating multiple tools from their IT environment. Since we had most of their essential capabilities like BitLocker management and remote control under one solution, MagnaMotors was able to cut costs and facilitate easy technician access.