SecOps integration: Bridging the divide between ITSM and
IT security

Interactions with ITSM practices

With incident management, IT admins can track, and manage the issues arising out of these patches on unique endpoints, or to a trove of assets. With the incident management approach, as the issue gets resolved, the resolution or work-around gets added to the knowledge base as a part of the process. An incrementally built knowledge base from incident resolutions further reinforces the patch management process with archival insights.

To maximize success with patch management, the patches are deployed through systematic process with change and release management. The change enablement practice, along with release management alerts the relevant stakeholders, mandates the plan for action, as well as sets approval gates, and rollback plans. In other words, change enablement and release management, together govern the what, when, and how of the patching process.

CMDB keeps tabs on the latest patch deployed on the configuration items (CIs). CMDBs also offers IT admins the infrastructure baseline configuration information that helps determine the necessity for the patches. IT asset management helps IT admins monitor the health of the assets post the deployment processes.

Equip your ITSM platform

Ensure the following requisites in your ITSM platform to build a strong bridge between your IT service operations and the security practices to strengthen your patch management activities

• The ITSM platform should have an integration within its incident management module that allows scanning the endpoints, and deployment of patches when necessary during incident diagnosis and resolution stage. This integration benefits IT teams in two ways: first, the productivity of the technician increases when the appropriate patches can be directly accessed within the ticket workspace. Second, patches can sometimes fail in specific workstations or server environments, despite the prior testing stage. The integration can allow an opportunity to check the status of the patch deployment in specific endpoints, and to validate if the patch if necessary.
• Change management and release management modules should be able to create dedicated workflows for each class of patches, with specific planning, implementation, testing, deployment, and review stages. Once approved, IT admins should be able to deploy the patches to the network as a whole from a single console.
• With built-in IT asset management, ITSM tools should house all the in-depth information of all assets present in the network. The integration with a patch management system should be keeping the asset inventory up-to-date on the latest deployments to each service component.
• ITSM tools require a CMDB to assist with patch deployment planning and implementation. CIs should be able to update themselves from the patch management systems, and provide the necessary details during a change implementation. Apart from identifying the current infrastructure state of patch compliance, the CMDB relationship maps help assess the level of impact, and radius of deployment needed during a change process.

Case in point

A supermarket chain, Zylker, had its point of sale (POS) systems unable to access the internet for nearly 12 long hours in locations across the country. Zylker's POS machines were not updated frequently because the IT admin team felt the POS system was highly stable and it felt regular reboots and the related downtimes were unnecessarily disruptive to business operations. However, the recently appointed chief security officer (CSO) did not feel comfortable with the high level of security risk this posed for the organization. She mandated all POS machines, and service components to be compliant with the latest patches.

To comply with this request, the IT admin in charge began with the firewall firmware upgrade. With several rounds of patches to deploy, the firewall upgrade seemed like a standard patch deployment process, and the IT admin proceeded to deploy it to all the systems. Unbeknownst to the IT admin, the patch reconfigured the content filtering engine in the operating system. This led to all internet access being blocked in every POS system. Resulting in a longer downtime for the cash counters in the supermarket.

Even after trying to roll back the upgrade, and attempting to patch the systems again after configuration changes, a few of the systems were shut down and couldn't be rolled back to the original version, leading to a failed rollback. This further increased the time to return to a fully operational status.

Lessons learned

• Initiate a change process no matter the severity of the patch: With a change process now, Zylker documents its deployment plans. This enables the IT team to verify whether the patch would work fine with comprehensive information on each asset, through the built-in IT asset management. These details are added to the change request to keep all stakeholders informed.
• Refer to the CMDB map to determine the scale of impact: Despite the low probability of firewall upgrades going wrong, Zylker cross-referenced the infrastructure map to ensure that if things go south, they have a reliable backup plan. The CMDB relationship map is associated with change request as well, to ensure stakeholders make decisions with the holistic view of the infrastructure in mind..
• Test, review, repeat: Zylker's has an integrated change and release process, which includes a dedicated user-acceptance testing, and reviewing stage. This ensures the organization is prepared to deploy to a subset of shops, check the results, reconfigure, and then move forward to another subset of shops. While this might seem to take more time than the previous "launch and forget" approach, it provides a fool-proof way to deploy patches, and document the entire process.
• Manage incidents from stray systems: A few of Zylker's stores previously had to shut down their systems during the rollback and repatch process. With an integration of its ITSM tool and its patch management tool, Zylker's IT was able to pinpoint these IT assets, create tickets for them, and deploy the latest firmware manually. This process is much more efficient for Zylker.

Interactions with ITSM practices

These are few of the scenarios where access management interacts with ITSM practices:

Incident management interacts with access management, such as a response to any unexpected certificate expiry or anomalous user behavior. Incident response process gets triggered for suspicious user behavior across the network. When a user's privilege score (a base score derived from the user role and historical behavior) drops due to accessing resources that are generally not accessed, an incident is raised, and the team begins analyzing the issue. Further, SSH key management interacts with incident response teams by securely connecting technicians with servers or workstations remotely.

Change management and access management are intertwined in their operations. During an implementation of a change, privilege management systems validate whether the change owner, manager and the implementation team have the authorizations to make changes in the respective CIs. Also, when any policies changes are made regarding user access and privileges, they are implemented via a change management process.

Request fulfillment is part of the access management best practice, where SSH keys and passwords need to be rotated in a regular schedule. Also, identity certificates need to be renewed before expiry. These are taken up as requests by the IT service desk team, and fulfilled as part of the security maintenance and upkeep. Apart from these, depending on the organization structures, end users sometimes request access to critical resources through the request fulfillment practice. , Privilege access requests are
fulfilled as well.

CMDB interactions are primarily around maintaining the records of the access privileges, keys, and identity certificates that each CI holds. Access management systems interacts with the CMDB to update them on a schedule.

Equip your ITSM platform

Ensure the following requisites in your ITSM platform to strengthen your access management activities:

• The ITSM platform should integrate a privilege access management (PAM) solution to ensure key rotation, vulnerable certificates detection, and certificate renewals are part of automated workflows in request fulfillment practices.
• During incident diagnosis, PAM solutions should allow ITSM platforms to initiate secure remote sessions to workstations by automatically fetching the SSH keys from the PAM database. These remote diagnosis sessions need to be recorded and added to the PAM system for security audits later.
• As part of Incident diagnosis, IT technicians should be given the least necessary privilege when accessing an application in the remote session. This application-level sandboxing should be possible with an integration between the ITSM and PAM systems.
• ITSM platforms should have built-in CMDB which houses the latest certificates associated the relevant CIs, fetched from the PAM system.
• The change management process can also be validated by an integration with the PAM system. Access to CIs for any changes would be granted only with an associated change that has been approved by the stakeholders.

Case in point

Zylker Health, a health care facility, faced a data breach of millions of records of electronic patient health information (ePHI). The breach occurred to a outdated SSL protocol, which served as the initial attack vector. Zylker's IT maintains a schedule to renew its SSL certificates, and manually updates protocol whenever necessary. However, due to the manual updates, they were exposed to human errors. Subsequently, one of the network devices was still using a SSL protocol version that was vulnerable to the "heartbleed" vulnerability. This went unnoticed and unpatched during their manual patching process.

Using the vulnerability, the attackers were able to glimpse over the credentials from memory, and used them to login to the network. With access to the network, they were able to exfiltrate patient records.

Lessons learned

• Maintain a certificate dashboard: Be on the lookout in your environment for any vulnerable or expired certificate that could pose a security risk. The best key management solutions offer a real-time dashboard of certificate authorities, vulnerabilities, expiries, and key summaries.
• Automate vulnerability scans of keys and certificates: Despite a low cadence of protocol updates, investing in an automated key and certificate environment scan can expose unforeseen devices posing as the weakest link in the chain.
• Set up a workflow to automate certificate renewal: Integrate the ITSM platform with your key management solution to automatically create incident tickets on any certificate expiry, or any exposed vulnerability. Treating such vulnerabilities as information security incidents with the same level of response ensures a better security posture.

Second Case in point:

Threat actors can arise externally or even from internal disgruntled employees. Such was the case with a small town's water management corporation. On one afternoon, the quality monitoring team (QMT) began to notice rising level of metallic minerals in the drinking water reservoir. Despite the multiple levels of filtration, the water quality was reaching lethal levels of contamination. To exacerbate the issue, the reservoir was open and was supplying the water to the town. An emergency alarm was triggered by the QMT and the reservoir was shut down. It took the decontamination team two days to get the whole system up and running again. After investigation, it was revealed that a recently offboarded employee still had access to the filtration system, and had changed the threshold limits in the systems without raising any alarms.

Lessons learned

• Automate privilege revocation: When the employee offboarding is automated on ITSM platform integrated with the PAM system, the water management corporation can be assured that employees are revoked of all access. With employee offboarding often raised as a ticket to the IT service desk, this automation eliminates the human errors that can result if security lapses.
• Set incident triggers on suspicious behavior: PAM system can analyze and set a baseline score on user behavior. When a user accesses resources at odd times, and critical systems that they might not need to, the trust score reduces for the user. Once the threshold is hit, an incident is automatically triggered in the ITSM platform alerting the incident
  response team.

Interactions with ITSM practices

Incident management and Problem management both interact closely with information security management. For instance, when an behavior anomaly is detected in the network, the incident response team (IRT) is immediately alerted. The IRT begins the triaging, diagnosis and resolution in a systematic manner, based on incident management best practices. Response to incidents needs to be a repeatable, measurable, and instructable process, otherwise chaos and confusion could impact which team to alert, the roles of each stakeholder, and sometimes which approach to take in resolving the incident.

Problem management eliminates the root-cause of an incident. Frequent occurrences of small issues, and any resource draining incidents, is addressed by the problem management team to break down the issue, find the root cause, and apply the solution. Problem management is a crucial supplement for the security posture of the organization. Log management tools with UEBA, can identify the recurring patterns of alerts, in the enterprise's IT landscape, and notify the problem team.

Change enablement interacts both directly and indirectly with information security management. Large enterprises have many changes being pushed to the infrastructure daily, and a poorly configured change can easily slip by to the infrastructure. These poorly configured changes can expose the system to unpatched vulnerabilities or security lapses. SIEM tools alert security teams on configurations changes to CIs that are unapproved, or might increase risk to the security posture of the organization. Indirectly, change management can be the result of problem resolution or an incident diagnosis warranting a change in the infrastructure. As such, when change processes are followed for every change being pushed out, it helps to maintain the established security plans and policies of the organization.

IT asset management and CMDB interactions with information security practice are quite similar. SIEM tools detect the configuration of the CIs, and determine its current state in its lifecycle, thereby keeping the inventory and CMDB up to date. Some organizations even use SIEM tools to verify CIs if the change has been implemented successfully as well.

Equip your ITSM platform

Ensure the following requisites in your ITSM platform to strengthen your information security management practice:

• The ITSM platform requires an integration with an SIEM tool, that enables automated ticket creation to manage incidents. The SIEM tool with UEBA, should alert the incident response team when any anomalous pattern is detected.
• The ITSM platform should also have an integrated problem management module that can drill down to find the root cause of any aberrant behavior from the IT environment.
• As a subsequent step from managing problems, ITSM platforms need a change management module, where a request for change can be raised to make a scheduled configuration change such as a firmware upgrade, or clear the database on servers, etc. based on signals received from the monitoring system. The change module should be able to block a designated CI within a specified time frame, so unauthorized changes are not allowed. Further, the change module of the platform should have workflow with built-in approvals, to ensure necessary security clearances are passed before implementing the change.

Case in point

The CEO of Zylker Logistics, one of the world's leading shipping companies, receives a call from the enforcement agency that its customer data has been leaked. The CEO panics, and checks with their IT manager immediately, to be met with poker faces from the team. The CEO was informed that the leaked customer data has information as old as 14 months, indicating that Zylker Logistics had been breached 14 months ago, and wasn't aware of it.

To trace and analyze how the threat actor breached the perimeter, the IT team filter, and sift through millions of rows of data from the log management tool to stitch together the log trail that led to the breach. After a long time, they figure out that Zylker was compromised through a malicious attachment in an email. The attachment had installed remote access software on a compromised laptop. The installation then reached out to the threat actor, who runs a shell session in the compromised endpoint, and is enabled to scan all the network devices, operating systems, and open ports.

Unfortunately Zylker hadn't patched a vulnerability on a server, which the threat actor uses to gain device access. The server access makes it easy for the threat actor to pull the domain admin's password hash from the memory. Using the hash, and subsequent access to the domain controller, the threat actor makes a copy of the password hashes. With the help of password recovery applications and a brute-force approach, the threat actor cracks the password and creates a personal account. This personal account has been granted nearly all access, except for the administrator level, which ensures the threat actor doesn't raise any alarms. Over several weeks, the threat actor siphons customer data over port 80/443 to an external server, and all of these activities are undetected.

While they received several alerts over the 14 months from separate monitoring tools, they did not correlate alerts together to infer upon the attack. Further, without any alerts being sent to the incident management team, there weren't any diagnosis or investigation for any of the unique alerts.

Lessons learned

• Configure anomaly detection using UEBA based solutions: Having just a log management tool, Zylker could not correlate several alerts together to infer a security breach. For instance, they received a port scan notification from one of the switches, but it was easily shrugged off as a traffic spike. Zylker's IT team did not piece together the port scan notification with the antivirus alert when the threat actor accessed the compromised workstation. A UEBA solution can establish a baseline behavior of users and machines, and flag abnormal behavior within an organization.
• Record alerts as tickets in the ITSM platform: For faster response to information security incidents, integrate the SIEM tools to the ITSM platform. SIEM's high-priority alerts would be raised as major incident tickets, which will immediately alert the security teams to be on their toes. With UEBA or basic correlation rules, the SIEM tool continually raises high-risk alerts to the IT service desk, as the attack progresses overtime. Post the alert, when a port scan and antivirus alert from the same IP address is reported, subsequent high-risk alerts are sent when hashed credentials are accessed, and an account is created by an unusual entity. Attacks are rarely undetected or unreported when a SIEM and ITSM tool are both utilized by an organization. A faster response results and mitigation actions can be taken quicker.
• Initiate root cause analysis to eliminate future lapses: It can take the Zylker's IT team many hours to scrutinize the logs and determine a root cause.. Once the security incident is resolved, a problem management ticket has to be raised. The problem management team needs to have usable insights generated from the ticket to drill down to the issue fast. SIEM tools, or log management tools, that can segment and filter out data, come to the rescue by assisting the IT team with analysis. UEBA-based solutions accelerate the root cause analysis, and provide its correlation data to guide the problem management team to a solution very efficiently..