Research on Network Security Situational Awareness Based on Crawler Algorithm

Network security situation awareness is a critical basis for security solutions because it displays the target system's security state by assessing actual or possible cyber-attacks in the target system. Aiming at the security and stability of global information flow, this paper studies the perception and measurement of the overall situation of network security. Through the Scrappy web crawler framework, data were collected from several Zhiming network security event websites, and based on the vulnerability database of China Computer Network Intrusion Prevention Center, the network security event database was designed and established, which enriched the data of situational awareness research. This study investigates the analysis and processing of network security events, a crucial parameter in the stage of security insight and perception, and builds and implements a text-based network security event analysis tool. By designing a network security event analysis tool based on text processing, the data cleaning of network security time text information is completed, and a set of network security event processing solutions with high applicability and comprehensiveness are formed. Statistical experimental results show that the network security event database built based on the crawler algorithm contains 43,848 pieces of data, which increases the capacity by 12.79% and 29.33% compared with the traditional algorithm, and reduces the reading time by 63.5% and 87.2%.