Practical data mining and analysis for system administration

Modern networks are both complex and important, requiring vigilant system administration. System administrators employ many tools to aid them in their work, but still security vulnerabilities, misconfigurations, and unanticipated device failures can occur regularly. Theconstant and repetitive work put into fixing these problems often leads to wasted money, time, and effort. We have developed a system to greatly reduce this waste. By implementing a practical data mining infrastructure, we are able to analyze device data and logs as part of general administrative tasks. This allows us to track security risks and identify configuration problems far more quickly and efficiently than conventional systems could by themselves. This approach gives system administrators much more knowledge about and power over their systems, saving them resources and time. The system is practical because it is more straightforward and easier to deploy than traditional data mining architectures. Generally, data analysis infrastructure is large, expensive, and used for other purposes than system administration. This has often kept administrators from applying the technology to analysis of their networks. In contrast to this we propose a system designed to overcome these problems.

We propose a lightweight, easily configurable solution that can be setup and maintained by the system administrators themselves, saving work hours and resources in the long run. One advantage to using data mining is that we can exploit behavioral analysis to help answer questions about points of failure, analyze an extremely large number of device logs, and identify many device failures before they happen. Indexing the logs and parsing out the information enables system administrators to query and search for specific items, narrowing down points of failure to resolve them faster. Consequently, network and system downtime is decreased. In summary, we have found in our tests that the system increases security response time significantly. We have also found that the system identifies configuration problems that had gone on unnoticed for months or even years; problems that could be causing many other issues within the network. This system’s ability to identify struggling devices by early warning signs before they go down has proven invaluable. We feel that the benefits and simplicity of this system are significant enough to make it worth implementing in almost any professional computer network.

Download paper here

Recommended citation: Lund, T., & Panike, H., & Moses, S., & Rowe, D. C., & Ekstrom, J. J. (2015, June), Practical Data Mining and Analysis for System Administration Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24570