Modern companies often face the need to migrate their information systems. However, this procedure must be preceded by careful preparation, since there are many obstacles along the way.

There can be a great many reasons for starting the transition to a new information system (IS), including reducing the risks associated with the operation of outdated platforms, bringing information systems to international standards, and increasing the efficiency of business processes. But no matter what task the company faces, the transition from one IP to another must be carefully planned and prepared.

Migration problems

When it comes to migrating transactional systems such as ERP, billing, processing or core banking, the transition to a new system is very problematic. The fact is that IT professionals need to ensure the accurate migration of large volumes of data, maintaining parallel operation of the old and new systems for reconciliation and analysis of the results.

For example, I had project experience in one of the largest banks, where a transaction system was being transferred from the no longer supported Informix platform to the Oracle platform. At the same time, it was necessary to conduct a thorough analysis of business processes, repeatedly transfer data from the old system to the new one, and check the consistency of the results of the new and old systems, taking into account the duration of the process regulations. That is why the migration period was 14 months. Sometimes parallel operation of two systems can continue for a longer time, but even when it is limited to several months, ensuring the operation of the new IS requires the allocation of additional computing power and significant time of enterprise employees to simultaneously perform tasks in two systems.

From department system to enterprise level

IP updating often occurs within the framework of globalization and centralization. This allows you to significantly reduce the cost of supporting and updating software systems. Indeed, maintaining a single platform that serves all employees is much easier than maintaining separate tools for each department. For example, successful migration of an inventory accounting system allows you to transfer several thousand departments of a large organization to a single platform and provide a serious reduction in IT costs. However, it should be remembered that most of the preparation in this case falls on the coordination of data in various formats and representations, the development of new regulations and the construction of new models of employee interaction.

Another important aspect is integration interfaces with other enterprise information systems, especially self-written and specific ones. Problems associated with them may not be so noticeable at the first stage, but are identified when establishing interaction between various departments and the overall system. And if for the old system such interfaces were already implemented programmatically or organizationally, then for the new system they may have to be developed anew.

It should be remembered that thoughts about expanding the functionality of the system can come already during the implementation of the project, just like appetite comes during a meal. This means that a whole series of additional work will be required.

Action plan

The experience of project activities on system migration shows that any such project requires careful preparation and must be accompanied by an individual plan. However, regardless of the type of systems being migrated, software, database volumes, etc., the general scheme looks almost identical.

At the first stage, it is necessary to conduct a detailed audit, finding out all the requirements for the mode of operation of the new system, interviewing all key users. It is important to understand what volumes of data and what kind of load we are talking about, only then will specialists be able to propose the right migration strategy.

The procedures themselves must also be carefully thought out and include such important elements as the rules for user access to systems during migration, procedures for rolling back to a previous state in the event of failures, and the interaction of the various specialists involved in these processes.

After agreement with the customer, a detailed plan is usually drawn up, which involves several stages, namely: data copying, verification, parallel operation of two systems and a complete transition to a new platform. In my opinion, the main thing in a professionally organized system migration is the smoothness of the process for users who can gradually, without stress, begin to work in a new automated system.

However, even careful preparation does not always save you from underestimating labor costs when transferring users to “new rails.” This process includes both training of company employees and their support during the period of adaptation to the new system.

Leif Poulsen for InTech

Systems for automating production and collecting information about production processes are relatively short-lived. They often have to be upgraded or replaced before the process equipment reaches the end of its life. For many companies, managing the replacement or upgrade of such automation systems without stopping production is a real challenge. Therefore, the objective need for modernization or replacement is ignored until something happens. This article is about how you can successfully accomplish this task through careful planning and organization.

Two main factors are behind the need to modernize and replace industrial automation systems and industrial IT systems: the technical degradation of these systems, as well as changes in the requirements of the business processes that these systems support.

The reliability of technical systems will decrease over time if companies ignore the need to modernize operating systems, databases, and application software. The operational risk of equipment failure increases accordingly.

Through careful planning, operational risk can be kept to an acceptable level, while also protecting investments and minimizing life cycle costs. For a typical automation or IT system, only 20-40% of the investment goes into purchasing the system. The remaining 60-80% goes to maintaining its high availability and adapting to periodically changing requirements.

In addition to assessing the activities required to prevent technical degradation, it is necessary to consider new challenges as well as potential business opportunities. The business environment is constantly changing, and all opportunities to improve existing or introduce new technologies must always be considered. Typical business opportunities that may drive the migration of high-cost automation systems are speed to market, competitiveness, growth, quality and regulatory compliance.

Long term migration plan

Developing a long-term system migration plan allows companies to maintain systemic operational risks at an acceptable level. In addition, it ensures risk management and timely support of business goals. The migration plan must take into account restrictions such as “best manufacturing practices”, technology functionality, and inevitable production downtime.

In general, the approach to long-term planning is depicted in Fig. 1. A migration plan is developed to determine where the company wants to be in five years, what actions need to be taken to get there, and whether the resources required to get there are available. This approach is based on the architectural design principles outlined in the TOGAF standard, which is widely used in the development of system architecture for industrial enterprises.

Figure 1. General approach to creating a long-term migration plan.

It is necessary to distinguish between the existing architecture and the target, desired one. The difference between them reflects the difference between the company's current position and the position it wants to occupy in the future. The migration plan charts the path from the existing architecture to the target architecture - possibly through several transition stages.

Each architecture can be described as a series of "layers" that bridge the gap between business and technology - as shown in Figure 1. 1. Attention must be paid to the following “layers”:

• Business goals it is part of the overall strategy planning effort. They allow you to choose the right direction of the process.
• Business model provides context in which production and business processes are understood. Typically, it includes a high-level description of material flows and processes.
• Description production and business processes is important for the successful application of technologies and the correct assessment of their value from a business point of view.
• Information, data and documents important for connecting processes and applications. Interoperability and management of information flows between applications are especially important.
• Descriptions applications allow you to formulate high-level requirements and define interfaces.
• Definition infrastructure, computing and network requirements (hardware, fault tolerance, performance).
• Provided services define requirements for ensuring effective operational management and decision support.

Developing a migration plan

Developing a migration plan for an entire organization, or even a single production site, can be a truly complex task involving many people. It is recommended to divide the development process into several stages, described below.

Part II

Stage 1: Mobilization

Basic goals:

• achieve a common understanding of tasks and goals
• mobilize the organization where the project is planned
• detail the plan by describing the milestones and results of the project phases
• collect all necessary/available information
• provide a proper understanding of concepts, practices and theory

• scheduled meetings
• workshop dedicated to the start of the project

Results:

• detailed consultation plan
• common goals
• process overview

Stage 2: Analysis

The goals of the analysis phase are:

• analysis of business and production processes in order to:

Assess the readiness of personnel servicing IT and automation systems

Understand the data and functionality needs for the future architecture

Identify key benefits of the future architecture to set goals and implement the business case

• analysis of existing architecture

Determination of existing production processes in their connection with automation systems, data collection, production management systems

Identification of existing business processes and their connections with production automation systems

Identify existing applications, data, logical and physical infrastructure, and technical support services

During this stage the following activities are carried out:

• seminars and discussions on various processes
• site visits to obtain contextual information
• seminars and discussions on existing systems
• assessment of services to determine their level of maturity and compliance with regulatory requirements

Results:

• identification of existing infrastructure
• analysis documentation
• list of ideas on the challenges and opportunities of the new architecture list of ideas on the challenges and opportunities of the new architecture

Stage 3: Goal

The purpose of this stage is to identify and describe the needs formulated during the analysis stage.

The solution, or target architecture, will describe:

• future business processes and functionality
• target application types, with their functionality, users, information and interfaces
• infrastructure needs and revised support standards

During this stage the following activities are carried out:

• seminars and discussions on process improvement
• workshops and discussions on improving architecture

Results:

• future architecture (presentation)
• Brief Description of Application Types

Stage 4: Justification

The purpose of the justification phase is to provide an initial business case based on rough estimates of the costs and benefits of the project.

The gap between the existing and the desired situation usually leads to the emergence of a number of ideas. Justification of ideas will allow you to distinguish “necessary” from “desirable”, and, after that, present and develop ideas to top management.

During this stage the following activities are carried out:

• rough estimate of costs and benefits
• first version of the presentation

Results:

• common goals
• prioritization of business ideas
• assessment of required resources

Stage 5: Plan

The purpose of this stage is to plan the project based on priorities, resources and dependencies:

• planning the sequence of implementation of the stages of a consolidated project
• providing the resources and competencies needed for the next steps
• initiation of project management activities
• completion of consulting and transfer of results of all stages to the customer

During this stage the following activities are carried out:

• development of an implementation plan
• development of an investment plan
• risk assessment

Results:

• implementation plan
• assessment of the workload of personnel involved in the project
• project risk assessment
• investment plan (as a first approximation)
• final version of the project presentation

Case Study

The following example illustrates the application of the described approach in real conditions. To comply with confidentiality conditions, anonymity is maintained in the description. We are talking about a fairly large enterprise that produces active ingredients for pharmaceutical products. The production facilities were commissioned more than 20 years ago, and although some modernization has been carried out since then, a number of outdated systems require replacement. Building automation systems and DCS are in the first place, as they are based on outdated technologies that are difficult to maintain. In addition, production must adapt to new business demands, including the discontinuation of some products and the launch of others. In general, there is a need to work on a migration plan that covers both technical and business requirements.

First, you need to create a master list of equipment that is currently used throughout the enterprise. This information is often “hidden” in various documents (and employees’ memories). It must be extracted and visualized so that it becomes the basis for migration planning. For this purpose, we typically create a Process Module Diagram showing the major equipment and raw material movements in each manufacturing unit. As separate layers "on top" of the hardware, we show which systems support which hardware.

An example is shown in Fig. 2. Data about installed systems is also contained in the system storage (or simply in Excel files), and can be used for further analysis and planning.

Figure 2. Automation “layer” allows you to evaluate existing systems

Before discussing a migration plan, it is necessary to identify the main business reasons for changes in production. In this case, management identified the following motives:

1. Consistent and error-free compliance with regulatory requirements

2. Minimum time required to enter the market, flexibility

3. Success, competitiveness, operational excellence

4. Uncompromising quality

5. Growth in production volumes

These goals need to be translated into more specific tasks, the implementation of which can be quantified.

Next, we must find out how well existing systems support current and future business processes. To do this, we use a standard reference model (based on the ANSI/ISA-95 series of standards). It includes 19 high-level business processes, detailed to the extent that it allows you to see weaknesses in their practical implementation and the need for change for the sake of effective business.

In addition, we also need to evaluate the technical capabilities of existing systems to support business processes in the future. This is done systematically, using the information described above from the system storage. For each system for which information is in the repository (in our case, about 70 systems), the following aspects need to be assessed:

• Equipment condition (failure history, mean time between failures, equipment age, spare parts availability)
• Status of the software (vendor support, availability of documentation, personnel with the necessary competencies)
• System recovery capabilities (redundancy, average service life before repair)
• Business impact assessment (information provision, data errors, unavailability)
• Indicative indicators (system reliability, critical importance of the system, etc.)

The technical assessment identified the need to modernize and replace a number of systems:

• Process control systems are based on a conventional, outdated DCS and many different PLCs, some of which are already “ripe” for replacement.
• The building automation system is based on a newer platform, but also requires upgrading to meet new requirements.
• A number of secondary systems also require modernization, or even replacement.
• The infrastructure serving all systems requires better segmentation and protection to meet today's security requirements.

Part III

After analyzing the business goals for the future, it became obvious that none of the existing systems fully meet future needs. This understanding gave rise to a number of ideas regarding the introduction of new technologies, as well as the production execution system. As a result of the analysis, 16 different projects were proposed that, if implemented consistently, will help the company meet future technical and commercial requirements.

The content of technical work and the cost of each project are assessed; For each project, a short one-page summary is prepared for management to discuss. (See Figure 3).

Rice. 3. One-page description of a potential migration project

In order to select priority projects, the potential results of each of them are assessed. The results are assessed in terms of business goals, as well as the reliability of the process control system.

Typically, you will need to evaluate multiple implementation scenarios to estimate the overall resource and funding requirements for each plan (Figure 7). One of the main constraints to consider is the windows in the production process during which systems can be replaced or modified. As a rule, these “windows” occur on weekends - and this is a serious bottleneck.

Rice. 7. Consolidated overview of the migration schedule

Since there is always little time to replace systems and set them up, preparation must be very thorough. Everything must be planned out in detail. An important aspect of planning is testing the systems implemented.

In the case we describe, the implementation of the long-term migration plan was carried out in six different streams, see Fig. 8.

Rice. 8. Organizing migration projects in six different streams

Part of the preparation is a thorough assessment and prevention of project risks. In Fig. Figure 9 shows typical risks associated with migration projects.

Rice. 9. Assessing typical risks of migration projects

Business Support Processes

The lifecycle management and long-term migration planning approach described in this article is driven by business needs. It includes an assessment of current and future business goals, as well as a thorough analysis of how technical systems will be maintained or replaced to best support those goals. The approach is based on TOGAF principles, which provide for sequential project implementation depending on the availability of budgets and qualified personnel. Assessing current and future system architectures is a key element in determining future migration projects. Finally, it is necessary to adhere to organizational change management principles that ensure the timely involvement of key project stakeholders, which is so important to the success of migration projects. The effectiveness of this approach has been repeatedly demonstrated in practice.

Leif Poulsen) ( ), leading automation and IT specialist at NNE Pharmaplan. He holds a master's degree in process management. At NNE Pharmaplan, Poulsen is responsible for the development of technologies, methods and competencies in the field of industrial automation and IT, and works as a senior business consultant.

Last updated: 10/31/2015

Often a situation arises when the model changes. For example, we decided to introduce new properties into it. But at the same time, we already have an existing database that contains some data. And in order to update the database without loss, ASP.NET MVC offers us such a mechanism as migrations. For example, we have a simple User model:

Public class User ( public int Id ( get; set; ) public string Name ( get; set; ) )

Accordingly, there is a data context through which we work with the database:

Users(get;set;))

And let’s say we have all the infrastructure for working with this model - views, controllers, and we already have several objects of this model in the database. But at some point we decided to change the application model base. For example, we added another field to the User model:

Public class User ( public int Id ( get; set; ) public string Name ( get; set; ) public int Age ( get; set; ) )

In addition, we decided to add one more model, for example:

Public class Company ( public int Id ( get; set; ) public string Name ( get; set; ) )

Thus, our data context is already changing as follows:

Public class UserContext: DbContext ( public UserContext() : base("DefaultConnection") ( ) public DbSet Users ( get; set; ) public DbSet Companies ( get; set; ) )

We can add an additional field for the Age property to the views for the User model, we can create a controller and views for the new model, but when we try to add a new object to the database, we will receive an error:

The data context has changed, and now we need to migrate from the old database schema to the new one. And first of all, find the Package Manager Console window at the bottom of Visual Studio, enter the command in it: enable-migrations and press Enter:

After running this Visual Studio command, a Migrations folder will be created in the project, where you can find the file Configuration.cs. This file contains a declaration of the Configuration class of the same name, which sets the configuration settings:

Namespace MigrationApp.Migrations ( using System; using System.Data.Entity; using System.Data.Entity.Migrations; using System.Linq; internal sealed class Configuration: DbMigrationsConfiguration ( public Configuration() ( AutomaticMigrationsEnabled = false; ContextKey = "MigrationApp.Models.UserContext"; ) protected override void Seed(MigrationApp.Models.UserContext context) ( ) ) )

In the Seed method, you can initialize the database with seed data. Now we need to create the migration itself. There, in the Package Manager Console, enter the command:

PM Add-Migration "MigrateDB"

Visual Studio will then automatically generate a migration class:

Namespace MigrationApp.Migrations ( using System; using System.Data.Entity.Migrations; public partial class MigrateDB: DbMigration ( public override void Up() ( CreateTable("dbo.Companies", c => new ( Id = c.Int( nullable: false, identity: true), Name = c.String(), )).PrimaryKey(t => t.Id); AddColumn("dbo.Users", "Age", c => c.Int(nullable : false)); ) public override void Down() ( DropColumn("dbo.Users", "Age"); DropTable("dbo.Companies"); ) ) )

In the Up method, by calling the CreateTable method, the table "dbo.Companies" is created and its configuration is performed: creating columns, setting keys. And a new Age column is also added to the existing table. The Down method removes the column and table in case they exist. In fact, these methods are equivalent to the ALTER expression in SQL, which changes the structure of the database and its tables.

And finally, to perform the migration, we will apply this class by typing the command in the same console:

PM Update-Database

After this, if we look at the composition of the database, we will see that changes have been applied to it in accordance with the migration performed:

So, the migration is done, and we can already use the updated models and data context.

In this article, we would like to systematize our experience in carrying out data migration in large corporate projects related to the transition of Customers to work in 1C:Enterprise 8 configurations.

At the same time, the main emphasis in the article will be placed, first of all, on the technological component of the migration process. The organizational component is also affected, but to a lesser extent.

Terms and Definitions

Data migration is usually understood as a final sequence of work, a project aimed at a one-time mass movement of data from source systems (historical systems) to the destination system. At the same time, the exploitation of this data in the source systems ceases.

Data migration must be distinguished from data integration. Integration, unlike migration, is a permanent part of the IT architecture and is responsible for the flow of data between different systems and data stores - and is a process rather than a project activity.

The migration scheme in general looks like this:

Rice. 1

Historical systems- databases of the Customer’s company, which are planned to be completely or partially replaced when implementing a new system.

Receiver system- target system, arbitrary configuration “1C:Enterprise 8”.

Initial data- data downloaded from historical systems into a custom xls file format. In this case, the xls format seems to be one of the most convenient, since the ability to upload to an xls file is present in many accounting systems of “previous generations”.

As a modern alternative, it is possible to consider the xml file format as a transport.

There are also options for using an intermediate database.

Transformation, conversion- the process of converting source data into data for loading. Data transformation occurs in accordance with loading templates. The result of the transformation is the data to be loaded.

Download data- data intended for loading into the receiving system. This article, as well as the source data, considers the xls format.

Data templates for loading- description of data tables to be loaded into the target system.

Migration stages

Let's consider the process of preparing and conducting migration step by step.

The organizational stages of migration include the following points:

· Defining a migration strategy. At this stage, the Contractor and the Customer agree on the technology for carrying out migration work;

· Determination of the composition of the migration working group. The working group should include specialists from both the Contractor and the Customer who are sufficiently familiar with the operation of historical systems (on the Customer’s side) and the target system (on the Contractor’s side);

· Preliminary migration plan. The migration plan will be adjusted several times as the project progresses;

· Periods of dates for downloading data from historical systems, volumes of data. Data cut-off periods for migrations, dates of test and final migrations. This information can be attributed to the migration plan;

· Composition of data to be migrated. Reference data, classifiers, transaction data, balances, turnover, etc.;

· Issues of checking the quality, correctness and integrity of data during the migration process and at the end;

· Issues of rolling back to a previous state in case of failures.

Let's take a closer look at the technological stages of migration.

Rice. 2

1.Preparing data loading templates

The data loading template contains technical descriptions of the data tables to be loaded, algorithms and loading rules for the current template.

Each template generally targets one or more related tables on the target target system.

The template states:

· Description of all fields of the xls data file for download, including:

o Field name

o Indicator that the field must be filled in

o Example of filling out the field

o Note

· Description of the rules for loading the target system table based on the data to be loaded (queue in the case of several related tables, search algorithms for key fields, etc.)

· Description of filling the fields of the target system tables directly if anything other than transferring data “one to one” from a data file for loading is provided. Relevant for reference fields, for example.

During the work at this stage, the Contractor must also prepare a data file loader for loading. When working with xls files, this task is not particularly difficult.

2.Identification of data sources

This stage can begin together with the previous stage “1. Preparing data loading templates."

At this stage, the Customer’s specialists determine from which systems and what data can be downloaded. You should also determine what data Maybe may be needed.

As a rule, in large migration projects, identifying a complete exhaustive list of data sources can take quite a long time and occurs as work continues in subsequent stages.

There are often situations when, in order to further ensure the integrity of information, some data has to be transferred from printed sources (digitized) or even entered into tables according to the words of the Customer’s key employees.

However, at this stage you should try to identify as much necessary data as possible.

3.Uploading source data

The process of downloading data from historical systems can take quite a lot of time, especially if there are many systems, they are different and different divisions of the Customer are responsible for them. This point must be taken into account during test and final migrations.

The most convenient option seems to be uploading to xls files. Many older IT systems support this option.

There may also be options for uploading to csv format, dbf, xml formats and others.

It is worth noting that for one reason or another (security issues, for example), the Customer cannot always provide data downloads in full at this stage! Just a data structure and a few test positions. Thus, a situation may arise that during test and final loads, low-quality data will be detected in the source tables, which will lead to unplanned errors.

To minimize this problem, the volume of test downloads from historical systems should be agreed upon in advance.

4.Data mapping

Mapping (data mapping) - in general, the process of comparing data from historical systems and the receiving system. That is, the source data and the data to be loaded.

The mapping stage is the most labor-intensive stage and can take up more than 50% of all work on the migration task.

At this stage, the entire migration project working group is fully involved.

In the process of data mapping, it is necessary to distinguish the substages of table mapping and field mapping.

· Mapping of tables, or mapping of templates - comparison of tables of source data and data templates for loading. The match can be either 1:1 or N:N. As a result of this work, a table mapping registry is compiled and maintained. This sub-stage is necessary for the next sub-stage of field mapping and for monitoring the general state of affairs in mapping.

Group of 1C templates	Name of the 1C template	File name- source	Rules for generating a source file	Responsible	Status	Note
NSI	Sample_ Nomenclature	Nomenk latura.xls	Set selection in system N . Save to txt . Open in xls, columns are text . The first line is the header . Number of columns - 15 . Check number of lines in txt and xls . The sheet name is always "Sheet1"	Ivanov I.I.	at work

· Field mapping - mapping table fields within an already defined table mapping. The result of this work is a field mapping registry.

№pp

Cl. field

Required

1C template field name “Template_Nomenclature”

Description

Field name "Nomenclature.xls"

Filling algorithm

Code

Directory element code

Code

Name

Name

Yes

This group

Contains one of the following values: . 1 - for groups . 0 - for elements

If code length=11 characters and last 4 characters<>"0000", then this element is "0", otherwise the group is "1".

Full name

Directory element name

Name

If ThisGroup = 1, Then "", ElseIf ThisGroup = 0, then Name.

As part of this stage, possible work on data normalization should also be carried out.

5.Preparing transformation rules

Unlike the previous stages, this stage is technical and involves the work of the Contractor developer.

Based on the agreed field mapping registers, the Contractor’s specialists develop rules for data transformation.

For operational work during the preparatory stages of migration and further, during test and final migrations, it is important that there is a convenient environment for developing rules (scripts) for data transformation and an environment for converting source data into data for loading.

The requirements for this environment include:

· Convenience and speed of development of transformation rules;

· Speed ​​of data conversion. The input and output files can be hundreds of thousands of lines long!

· Ability to work with several input files simultaneously;

· Ability to save transformation rules in separate files.

For our migration projects, we have developed a specialized developer's workstation, using the standard 1C Query Console processing as a basis.

The Query Console processing has been improved to allow direct queries to xls files.

Here is an example of combining two source xls files Employees.xls

Employee code	Surname	Name	Surname	Date of Birth
2423	Ivanov	Ivan	Ivanovich	17.11.1992
1523	Petrov	Basil	Aleksandrovich	04.02.1991
4363	Sidorov	Kirill	Nikolaevich	01.05.1995
	Denisov	Denis	Denisovich	01.01.1990

And Operations.xls with pages:

Write-offs

Employee code	date	Sum
2423	01.02.2014
1523	02.02.2014
4363	03.02.2014
	04.02.2014	100000
2423	05.02.2014
1523	06.02.2014
4363	07.02.2014	2356
	08.02.2014	140000
2423	09.02.2014
1523	10.02.2014
4363	11.02.2014	23523
	12.02.2014	80000

And Receipts:

Employee code	date	Sum
	01.05.2004
	02.05.2004
	03.05.2004
	04.05.2004
2423Date of Birth	Receipt amount	Amount written off
Ivanov Ivan Ivanovich	2423	17.11.1992	1341234	1010
Petrov Vasily Alexandrovich	1523	04.02.1991	245245
Denisov Denis Denisovich		01.01.1990	380000	320000
Sidorov Kirill Nikolaevich	4363	01.05.1995	613382	26336
TOTAL:			2579861	347842

Note that the example is artificial, specially selected to demonstrate all possible stages of transformation of data sources.

The technological sequence of transformation operations here is as follows:

Using the Access SQL query language (which provides significant additional capabilities compared to the 1C query language), an initial query is created that extracts data from the xls file into the 1C environment. At the same time, various checks and normalization of data are already possible at this stage.

ADO data access technology provides high speed.

Rice. 3

2. Query in 1C language - the main query that implements the field mapping algorithm. And also: enrichment of downloaded data with data from the 1C database, regrouping, merging with the results of queries to other source xls files, etc.

3. Post-processing of the 1C request result if necessary. Implemented using a script in 1C language.

For example, here we implement the addition of the “TOTAL” line in the amount columns.

4.Write the final data set into an xls file.

In general, the output is final files for loading into the target 1C database.

This tool also allows you to save data conversion rules in a separate xml file:

In addition, it is possible to work V batch mode, which is especially important when there is a large amount of heterogeneous migrating data.

During the previous stages, the preparatory part of the work generally ends - all data sources are identified, source data is downloaded from the sources, download templates are prepared into the target database, data mapping is prepared and, finally, data transformation scripts are developed.

It should be noted that before the final migration you should definitely conduct several tests. During test migrations, the Contractor, together with the Customers, identifies:

Conversion errors, data loading errors

Conduct a preliminary assessment of the quality of data loaded into the target system

Based on the results of test migrations, they create/update a final migration plan

7.Data reconciliation

The quality of the downloaded data should be checked both after test migrations and at the end of the final migration. During the reconciliation, the following indicators can be checked:

· Coincidence of total amounts for balances, according to documents;

· Quantitative matches, for example the number of OS;

· Correct filling of individual selected entities;

Please note that certain checks of migrating data and issues of data normalization must be resolved throughout all migration processes. You must always ask yourself what needs to be done at the current stage in order to avoid mistakes in subsequent stages.

For example:

· Check for duplicates by key fields. It can and should be carried out on the original data;

· Coercion of field types;

· Referential integrity;

· Mathematical inconsistencies. For example, checking for empty numeric fields into which division is planned during transformation;

· In general, checking mandatory fields are filled in;

· Replacement of incorrect characters. For example, English characters in Cyrillic fields (“o”, “a”, “e”, etc.) This is especially true for key fields!

· Checking the values ​​of string fields for compliance with the types of the receiving system (Length restrictions)

After the final migration is completed, according to a predetermined migration strategy and migration plan, a decision is made on the further operation of the historical systems.

Often the operation is completed immediately after the final data reconciliations and recording the success of the migration - users of the new system no longer keep records in two systems in parallel, but completely switch to the new system. At the same time, access to the old system is maintained in read mode.

In some cases, parallel operation of two systems may occur for the duration of trial operation (TE) and even beyond this period. The issue of parallel work of users in two systems is closely related to the issue of the possibility of rolling back to the old system if the migration (or, in general, the operation of the new system!) is considered unsatisfactory.

Conclusion

In conclusion, I would like to note that when it comes to migrating large transactional systems, which include many 1C:Enterprise configurations, the transition to a new system can be very labor-intensive.

Therefore, it should be remembered that any such project requires careful preparation and must be accompanied by an individual plan. However, regardless of the type of systems being migrated, database volumes, etc., the general migration scheme looks almost identical.

• transfer existing resource domains into organizational units of new domains, which will simplify the management of network resources;
• “simulate” the progress of migration, while no real data transfer occurs;
• undo actions taken related to migration;
• move service accounts;
• restore trusting relationship between source and target domains;
• Convert multiple domains into one or more large domains in an already created Active Directory environment;
• restructure existing groups or merge several groups into one in the target domain;
• analyze the data transfer process by logging migration events.

Migration of users and workstations into a single Active Directory structure is carried out while preserving existing access rights.

Upgrade options

There are two main options for upgrading the domain infrastructure [4]:

• Domain update. This method is the most common and easiest to implement when migrating domains. This method allows you to save the current domain structure, system settings, user and group structure. Domain update (in-place update) involves transferring existing domain controllers to the newly created domain.
• Domain restructuring. This method allows you to change the existing structure of domains, merge domains, or convert domains into organizational units.

In addition to the above options, there is also a mixed option based on them - updating domains with their subsequent restructuring [13].

These options are called Transition Paths for Active Directory Implementation. The transition path chosen from them will be the main link in the overall strategy for updating the domain infrastructure. This strategy will include a description of which directory service objects need to be moved and in what order. The best practice for any application move during an Active Directory implementation is to document every detail in a working document called a transition plan.

Criteria for choosing a transition path

When choosing a transition path, it is assumed that the decision concerns only one domain, meaning that it is perfectly fair to use different transition paths for different domains within the same organization.

Let's consider the main criteria that are used when choosing the most suitable transition path [13], given in tables 12.1, 12.2, 12.3, 12.4, 12.5, 12.6.

• Criterion 1. Satisfaction with the existing model of the existing domain. Table 12.1. Selecting a transition path based on criterion 1

  Transition path	Eligibility criteria
  Domain update	If there are no significant changes that you would like to make to the domain model, then updating the domain will provide the easiest path. The domain name will remain the same, as will the existence of all user and group accounts
  Domain restructuring	If the current domain model no longer meets organizational needs or is no longer the best fit for the organization's departments, domain restructuring may be the best choice.

• Criterion 2. The degree of risk when moving to a new domain model. Table 12.2. Selecting a transition path based on criterion 2

  Transition path	Eligibility criteria
  Domain update	Upgrading a domain is a low-risk method. The domain controller upgrade process is automatic, so there is little room for error without user interaction. The methodology for recovering from a domain upgrade failure is also relatively simple: if the upgrade fails, you need to shut down the primary domain controller (PDC), assign any backup domain controller (BDC) that has fresh data to the PDC role, and start the procedure again
  Domain restructuring	Domain restructuring is a higher risk path than domain renewal. There are more tasks to complete, and therefore many processes may go wrong. As a result, there is growing frustration among users who are unable to log in, access needed resources, or access their mailboxes.

• Criterion 3. Transition 1 execution time The timing of the transition is not a deciding factor in choosing a transition path, but it may be a determining factor for small organizations with limited resources. .Table 12.3. Selecting a transition path based on criterion 3

  Transition path	Eligibility criteria
  Domain update	Domain renewal is a linear process: once it has been started, it must be completed. It requires fewer steps than a restructuring and therefore takes less time to complete the entire transition
  Domain restructuring	Domain restructuring always takes longer. For example, during a restructuring, a lot of time is spent building and validating the infrastructure of the target domain, moving all accounts from the source domain to the target domain. Large organizations may not be able to move all the objects at once, so quite often domain restructuring is done in several stages

• Criterion 4: Directory service time required to complete the migration process. Table 12.4. Selecting a transition path based on criterion 4

  Transition path	Eligibility criteria
  Domain update	Account objects are not available during the migration process because they are self-updated when the domain is upgraded
  Domain restructuring	A good choice for organizations in which system working time is a critical value. Because it involves creating an unpopulated, "clean" forest and leaves the original environment essentially unchanged, the functionality of the directory service is preserved as users continue to function in their existing environment. You can migrate large or small batches of users during off-peak hours and leave these new accounts dormant until you are ready to leave the old system

• Criterion 5. Availability of resources to complete the transition. Table 12.5. Selecting a transition path based on criterion 5

  Transition path	Eligibility criteria
  Domain update	Since the domain update is an automated operation, this transition path will require fewer human resources
  Domain restructuring	Domain restructuring entails more tasks than domain renewal and therefore requires more resources, meaning that it must be adequately staffed to handle the additional workload associated with domain restructuring. An alternative is to outsource some or all of the project: there are many consultancy groups that specialize in such projects, saving the time and money needed to train internal staff

• Criterion 6. Transition project budget. Table 12.6. Selecting a transition path based on criterion 5

  Transition path	Eligibility criteria
  Domain update	Factors contributing to a decrease in the required budget funds: ability to use existing server hardware; lower human resource costs; reduction in testing costs since fewer upgrade tasks will need to be tested
  Domain restructuring	For many reasons, domain restructuring will require a larger budget than domain renewal. The hardware requirements needed to build a bare forest environment to which directory service objects must be migrated should be considered from a budget perspective

If a company does not quite meet the conditions to confidently choose domain renewal or restructuring as the renewal path, or if both paths are suitable for it, then it can choose a third path - domain renewal followed by restructuring.

This path to Active Directory will provide immediate benefits (delegation of administration, group policies, application publishing and more), as well as the long-term benefits of domain restructuring (fewer domains with increased domain volume, domain design in accordance with the company's business and organizational goals).