Severstal develops biggest Data Lake in Russian industry

PAO Severstal, one of the world’s leading steel and steel-related mining companies, announces that it has created a hybrid Data Lake as part of its digital strategy to secure the Company’s competitive advantages in the long-term. The infrastructure is designed to store Company functional data files for subsequent processing and use in Severstal’s data analysis, machine learning and artificial intelligence projects.

The hybrid architecture of the storage will combine two data storage models – a private Data Processing Centre (DPC) and leased cloud services. The project will be based on open-source software.

Severstal has already signed a contract with Lenovo Group which will supply the servers. The private DPC will have 2 petabytes (PB) of storage. The pool will also have 30 terabytes (TB) of read-write (RW) memory and 1,200 core processing. This will enable the Company to create one of the biggest corporate Data Processing Centers owned by manufacturing companies in Russia.

In addition, Severstal has signed a framework agreement with Microsoft for the lease of computational resources within the Microsoft Azure cloud service. The agreement will enable Severstal to benefit from a hybrid model, whilst also giving the Company access the unlimited resources of Microsoft’s highly secure public cloud. The cloud supports a variety of provider technologies including open source. Severstal is planning to build dynamic data storage which will be used for projects that require specific resources over a specific time period.

A solution based on Apache Kafka and Spark open-source software will be used for data transfer; not only will this make it possible to transfer data at high speeds, but it will also analyse the data in real time.

Igor Bardintsev, Chief Digital Officer at Severstal, said:

“Data processing is at the heart of almost all aspects of Severstal’s digital transformation. Therefore, the creation of infrastructure that is able to store and analyze enormous data files generated by the Company is key to Severstal’s digital transformation. The data warehouse’s hybrid architecture will offer a cost-effective solution to all of the challenges we currently face relating to machine learning and predictive analytics, as well as high performance data transfer, processing, and IT security.”

Severstal’s Data Lake will store data collected from transmitters on industrial equipment (Internet of Things), process management information system servers and MES-systems. The stored data will support predictive analytics projects in areas including predictive equipment maintenance and product quality optimization as well as other areas where artificial intelligence can improve efficiency.

Igor Bardintsev added:

“The overall aim of all of Severstal’s digital projects is to achieve maximum process efficiency. We have already introduced several initiatives at CherMK, for example, a defect prediction project at the cold rolling shop; we have several models in pilot mode working on predictive maintenance at Mill-2000 and we are seeing positive results. The more data we can collect, store and process, the more process solutions we will develop. That is why data warehouse development is a continuous and ongoing process.”