Blue Team Manual ⑤ [Abnormal Traffic Volume] Analysis and Application of Abnormal Traffic Blocking Tactics

▌Definition and Hazards of Abnormal Traffic

Definition of Abnormal Traffic

In a network environment, traffic refers to the collection of data packets transmitted through network devices (such as routers, switches, etc.). Abnormal traffic generally refers to traffic that deviates from normal network behavior patterns. This traffic may be caused by hardware failures, malware propagation, or network attacks. Abnormal traffic is not only an important means of network attacks, but can also be caused by configuration errors or system failures.

Classification of Abnormal Traffic

• Communication Anomalies: Including traffic interruptions or fluctuations caused by device failures, line issues, user operational errors, etc. For example, traffic interruption caused by router port failure, or sharp fluctuations in network bandwidth caused by sudden events (such as hardware failures, device overload, etc.). It usually manifests as a sharp increase or decrease in traffic within a short period, such as a large loss of data packets or a significant increase in network latency.

• Malware Propagation: The propagation of malware (such as viruses, worms, etc.) in the network will lead to abnormal traffic. A typical manifestation is a sudden surge in bandwidth consumption, especially when worm viruses propagate, where malicious traffic will occupy a large amount of network resources, causing network congestion or even collapse. This kind of traffic anomaly usually breaks out intensively within a certain period of time and is accompanied by abnormal traffic on specific ports or protocols.

• Network Attacks: Network attacks (such as DDoS attacks, brute force cracking, port scanning, etc.) usually generate a large amount of abnormal traffic in a short period of time. These attacks not only feature high frequency and large volume, but may also deal a fatal blow to the target system. Attack traffic is widely distributed and has obvious periodicity or randomness.

• Data Transmission Anomalies: Including data transmission anomalies caused by improper configuration or accidental operations, such as repeated sending of data packets, data transmission failures, etc. These anomalies often occur when network protocols are inconsistent or the network environment is unstable, which will affect data integrity and transmission efficiency. Of course, it also includes hackers using communication protocols allowed in the target environment (such as HTTP, DNS, ICMP, etc.) for data exfiltration, in which case the characteristics of these communication protocols will differ from the characteristics of data transmission under normal circumstances.

• Other Anomalies: Other factors such as misoperations and system errors may also lead to abnormal traffic. These anomalies usually do not have obvious attack characteristics, but if not handled in time, they may cause more serious problems.

Hazards of Abnormal Traffic

The hazards of abnormal traffic to network systems are mainly reflected in the following aspects:

• Business Interruption: Abnormal traffic may lead to excessive consumption of network resources, thereby resulting in the interruption of normal business. For example, DDoS attacks often exhaust server resources, leaving legitimate users unable to access. Business interruption not only affects user experience, but also may cause significant losses to corporate reputation and economic interests.

• Security Threats: Abnormal traffic may be a precursor to attacks carried out by attackers, such as port scanning, brute force cracking, and other behaviors that may lead to system penetration, thereby causing data leakage or system collapse. Such threats are often hidden and highly destructive, requiring timely detection and defense.

• Performance Degradation: Abnormal traffic occupies a large amount of network resources, which may lead to network performance degradation, such as increased network latency and reduced throughput. These problems will affect the network experience of normal users, thereby affecting business operations efficiency.

▌Network Intrusion Methods and Phase-based Behavioral Analysis

Intrusion Methods in the Boundary Breakthrough Phase

The boundary breakthrough phase is usually the first step for an attacker to invade the target system, mainly through the following methods:

• Information Gathering: Attackers usually start with information gathering, with the goal of understanding the basic structure, open ports, running services, and other information of the target network. This information can be obtained through public resources, social engineering, network scanning, and other means. Attackers may use tools such as WHOIS queries, NSlookup, and scanning tools to obtain network information.

• Vulnerability Scanning: After obtaining basic information, the attacker will perform vulnerability scanning on the target network to find vulnerabilities in the system or applications. Common vulnerability scanning tools such as Nmap and Nessus can help attackers identify open ports and potential vulnerabilities. Scanning may include looking for unpatched software, open management ports, and unprotected services.

• Boundary Breakthrough: Once a vulnerability is discovered, the attacker will attempt external network penetration through the vulnerability to gain control of the server. Common penetration methods include SQL injection, cross-site scripting (XSS), file upload vulnerability exploitation, etc. For example, through SQL injection, attackers can access sensitive information in the database or obtain system privileges.

▌ Penetration Behaviors in the Intranet Lurking Phase

After successfully invading, establishing a foothold, and achieving persistence, attackers will usually attempt to further expand their scope of influence within the intranet, mainly through the following ways:

• Lateral Movement: After obtaining preliminary permissions on the foothold in the boundary area, attackers usually attempt lateral movement within the intranet to expand control over more devices and services. For example, using the compromised server as a stepping stone, the attacker can attempt to access other internal network devices or workstations.

• Privilege Escalation: Attackers may exploit other vulnerabilities or misconfigurations within the intranet to attempt privilege escalation. For example, escalating from common user privileges to administrator privileges to obtain greater control. Privilege escalation may be achieved by exploiting local privilege escalation vulnerabilities, cracking service passwords, or exploiting misconfigured services.

• Data Theft or Destruction: After escalating privileges, the attacker may begin to execute the ultimate goal, such as stealing sensitive data, destroying system files, or planting backdoor programs. Attack behaviors in this phase are usually very stealthy, and attackers will try to minimize the probability of being detected to prolong their control over the system.

▌Detection Methods for Abnormal Traffic

Detection Method Based on Fixed Thresholds

Fixed threshold detection is the simplest method for abnormal traffic detection. By setting one or more predefined thresholds, when network traffic exceeds these thresholds, the system will trigger an alarm or take blocking measures. This method is simple to implement and suitable for most common detection scenarios, but it has poor adaptability to dynamic changes in traffic, making it prone to false positives or false negatives.

Feature Value Matching Detection

Feature value matching detection identifies abnormal traffic by establishing an abnormal behavior signature database and matching current network traffic features with known signatures. It can effectively detect known attack behaviors, but is ineffective against unknown threats, and the maintenance of the signature database requires a large amount of resources.

Data Statistics and Analysis Method

By performing statistical analysis on historical data, a baseline model of normal traffic is established and compared with current traffic to determine whether anomalies exist. This method is suitable for detecting progressive abnormal behaviors, but has a strong dependency on historical data and lacks real-time performance.

Intelligent Detection Based on Data Mining and Machine Learning

Utilising data mining and machine learning technologies, detection models are generated and continuously optimized by analyzing and training on large amounts of traffic data to improve detection accuracy. This method performs outstandingly when facing complex and unknown attacks, but consumes significant computing resources and has poor model interpretability.

▌Web Working Mechanism and Security Challenges of Modern Applications

Web Service Model and Communication Flow

The Web service model is one of the most common architectures in the modern Internet. Users send HTTP requests through browsers, and Web servers parse data according to the requests and return corresponding resources or execute corresponding applications.

• Typical Communication Flow:

  • Users access Web pages through a browser and send HTTP requests.

  • After receiving the request, the Web server parses the URL in the request and processes it according to the resource it points to.

  • If the request is for static content (such as HTML, CSS, images, etc.), the server reads it directly from storage and returns it.

  • If the request is for dynamic content (such as querying a database, executing a script, etc.), the application server executes the corresponding business logic.

  • After execution is completed, the server packages the generated response content into an HTTP response and returns it to the client.

Modern Challenges of Web Security

With the development of Web technologies, the complexity of Web applications is also increasing, which brings more security challenges:

• Cross-Site Scripting (XSS): Attackers inject malicious scripts into Web pages, causing other users to execute these scripts when visiting the page, leading to information leakage or session hijacking.

• Cross-Site Request Forgery (CSRF): Attackers entice users to click on malicious links or forms, utilizing the user's identity to execute unauthorized operations, such as transferring funds, modifying account information, etc.

• SQL Injection: Attackers bypass the application's verification mechanisms by inserting malicious SQL statements into input fields to operate directly on the database, thereby stealing or tampering with data.

▌Design and Implementation of Multi-level Detection and Blocking Strategies

Basic Version of Abnormal Traffic Analysis System Architecture

To effectively address abnormal traffic, a multi-level detection and blocking strategy has been designed. The overall architecture of the system includes the following parts:

• Traffic Collection Layer: Responsible for real-time collection and mirroring of network traffic, providing raw data to subsequent analysis modules. Common tools include Packet Capture Tools (such as tcpdump, Wireshark) used to capture network packets, and NetFlow/sFlow used to collect network traffic metadata. Of course, a considerable proportion of enterprises have also deployed commercial Tap switch products to achieve more flexible traffic collection and orchestration.

• Data Processing Layer: Cleans, parses, and extracts features from the collected traffic data, providing structured data for subsequent analysis. Data cleaning is performed through Python or Logstash, and protocol parsing uses tools such as Wireshark or Bro/Zeek. Now, a considerable proportion of enterprises have deployed commercial full-traffic analysis products to parse packets and perform threat analysis.

• Anomaly Detection Layer: Identifies abnormal traffic through various detection technologies (such as threshold detection, behavioral analysis, machine learning, etc.). For example, fixed threshold detection can set traffic threshold rules in Suricata or Snort, while behavioral analysis analyzes traffic patterns through Zeek.

• Response and Blocking Layer: Takes corresponding automated response measures based on detection results, such as traffic limitation, IP banning, alarming, etc. Common tools include Ansible, SaltStack, etc., to implement automated response processes, combined with firewall rule updates and SDN (Software Defined Networking) technology for dynamic adjustments.

• Visualization and Monitoring Layer: Real-time monitors network traffic status and detection results through visualization tools such as Kibana and Grafana, and generates reports for further analysis and decision-making.

  

Advanced Version of Multi-layer Filtering Traffic Detection and Blocking System

The advanced version of the system, through hierarchical detection and response mechanisms, can monitor, analyze, and block various network threats in real time, protecting the organization's network environment and key business from attacks and intrusions. The system architecture is divided into the following seven layers:

1. Data Collection and Preliminary Filtering Layer: Collects traffic from various network areas, uses common tools for preliminary filtering, and classifies and labels according to traffic types.

2. Behavioral Analysis Layer: Utilizes baseline detection and Deep Packet Inspection (DPI) technologies to analyze traffic behavior, identify abnormal activities, and mark suspicious traffic for the next processing step.

3. Feature Matching and Advanced Detection Layer: Matches known threat signatures through IDS/IPS systems, combines machine learning models to detect unknown threats, and dynamically adjusts the rule base to improve detection accuracy.

4. Automated Response and Manual Intervention Layer: Uses automated tools for response while retaining manual intervention options to ensure flexibility in handling complex scenarios.

5. Blocking Strategy Layer: Implements IP banning, protocol banning, and application layer banning, and dynamically adjusts blocking strategies based on detection results.

6. Blocking Strategy Countermeasure Response Layer: Handles the countermeasure behaviors of attackers, combining multi-layer blocking and deception strategies to enhance defense.

7. Comprehensive Evaluation and Feedback Optimization Layer: Continuously collects data for evaluation, optimizes detection models and strategies, and ensures the long-term effectiveness of the system.

▌Practical Applications and Optimization Measures

Deployment and Practice of Abnormal Traffic Detection System

In practical deployment, the strategies of the abnormal traffic detection system need to be optimized according to specific business and network structures. Common deployment methods include:

• Boundary Deployment: Deploying the detection system at the network boundary to monitor external threats.

• Internal Deployment: Deploying within the core network or data center to monitor internal traffic anomalies and prevent internal threats from spreading.

• Hybrid Deployment: Combining boundary and internal deployment to achieve multi-level traffic monitoring.

Common Application Scenarios

Targeted detection and countermeasure measures have been proposed for specific network attack scenarios:

• Abnormal User-Agent Request Detection: Identifies and blocks requests generated by automated tools or scripts by detecting abnormal User-Agent fields. Technical implementation includes configuring custom rules in WAF, combined with rate limiting to filter high-frequency requests.

• Access Control Based on Geographic Location: Restricts access from specific countries or regions, especially when the target system only operates in specific areas (such as domestic). Technical implementation includes utilizing GeoIP plug-ins to determine the geographic location of access requests and automatically blocking IP addresses from high-risk areas.

• Abnormal Request Detection of a Single IP: By setting request frequency thresholds, when an IP exceeds the set request frequency, it is automatically banned and its behavior pattern is recorded for further analysis. Technical implementation includes IP-based filtering and banning through firewall rules and access control lists.

Optimization Measures

Based on the existing system, the following optimization measures can further improve detection efficiency and accuracy:

• Machine Learning Model Optimization: By introducing machine learning models, automatically learn and optimize detection rules to improve the capability to identify unknown threats.

• Automated Response Enhancement: Combined with AI technology, the automated response system can automatically adjust protection strategies based on historical data and real-time analysis results, reducing manual intervention.

• Comprehensive Threat Intelligence Integration: By integrating external threat intelligence sources, real-time update and optimize the signature database of the detection system to improve the capability to identify emerging threats.

▌Future Challenges and Development Trends

Challenges in Network Traffic Anomaly Detection Technology

Future network traffic anomaly detection will face the following challenges:

• Detection of New Types of Encrypted Traffic: With the wide application of TLS/SSL, more and more traffic is encrypted, and how to effectively detect abnormal behavior in encrypted traffic has become a major challenge.

• Complexity of Network Environment: Modern enterprise networks are usually composed of various different devices, protocols, and services, and detection systems need to adapt to this complex environment to effectively identify threats.

• Popularization of Zero Trust Architecture: With the gradual promotion of the Zero Trust security model, network protection will pay more attention to access control and fine-grained security strategies, achieving all-around protection from the inside out.

Integration and Development Directions of Emerging Technologies

• Deep Application of Artificial Intelligence: AI technology has broad application prospects in the field of cybersecurity, and future detection systems will rely more on AI for threat analysis and response.

• Combination of Big Data and Traffic Analysis: Through big data technology, it is possible to perform deep analysis on massive amounts of network traffic data to discover hidden abnormal behaviors and potential threats.

• Automated Response and Remediation Mechanisms: Future detection systems will further develop automated response technologies, capable of automatically taking control measures the moment a threat is detected to prevent the attack from spreading.

▌Building a Solid Network Defense Line

Abnormal traffic monitoring and blocking is an important part of cybersecurity protection. By comprehensively applying various detection technologies and strategic countermeasure measures, we can build a more solid network defense line to effectively respond to various network threats and challenges. We hope this article can provide valuable reference and learning for everyone, and jointly promote the development of the cybersecurity industry.

▶▶▶ For more detailed content, please download Blue Team Book ⑤ [Abnormal Traffic Volume] Analysis and Application of Abnormal Traffic Blocking Tactics to get the full report.