Exclude file of Rsync
In recent years, with the emergence of a series of new technologies and new ideas in the field of IT software, such as cloud computing, micro-service, container, cloud native, DevOps, etc. The popularity of the concepts of "IT drives business" and "IT leads business"; As well as the publicity of Internet companies led by BATJ in some activities, IT still has a certain impact on the IT department of the traditional industry. The IT department of the traditional industry should face such changes in different positions. How should we adjust to deal with them? This paper will share and discuss this issue from three posts: architecture, development and operation.
First of all, let's talk about the structure post. The problems facing the architecture post mainly include the following aspects:
One is the shift from traditional infrastructure to cloud-based infrastructure. Under the traditional architecture, system construction and hardware equipment were based on centralized thinking, and the architect's thinking was more about the vertical capacity expansion of system construction and equipment. As a result, the single application became bigger and bigger, and contained more and more functions. Even a fine-tuning would "break bones". The idea of hardware selection also focuses on the performance, stability and longitudinal capacity expansion of a single device. But we all know that the resource pool of "cloud computing" is mainly x86 servers that are far less stable than minicomputers, and the emphasis is on horizontal scaling capacity. "Poor stability" and "lateral expansion" are in direct 180 degree conflict with the original thinking. What should we do? First of all, it is necessary to make clear that "cloud" is layered. Generally, the classic division is IaaS/PaaS/SaaS layer, and it is a kind of service-oriented concept of resource allocation according to demand. Since it is a service, we should first see who our service objects are, and then design different architectural schemes according to different objects. For iaas layer, for example, his main service object is the operations staff and developers, so in the design of the iaas layer architecture, as an architect should seize the role of these two kinds of different requirements, operations staff pay more attention to how can provide a stable and efficient computing and storage services, can support the thermal expansion, can know that allocate resources use is reasonable, Whether it can support local double activities and disaster preparedness in other places; Developers focus on whether the IaaS layer can provide fast and even self-service delivery of computing resources, quickly recover my application if a disaster occurs, and so on. At the same time, especially during the outbreak this year, mobile office and unified desktop cloud is becoming more and more attention, the traditional enterprise's traditional post is not before, now as an architect, consider to support this demand, in the aspect of architecture needs to be done which adjustment (network access, bandwidth, current limiting, identity authentication, link encryption, data leakage protection and subsequent expansion, etc.).
The second is containerization technology. Container as a hot word in the IT field in recent years, especially after the unification of K8S, if you go out and don't talk about K8S and container with others, IT will be felt that you are lagging behind an era. After the introduction of containerization technology, it is necessary for the architect to consider whether the container should be deployed on a physical machine or a virtual machine, whether the container should be deployed on a separate network segment or shared with the original network segment, which applications are suitable for deployment on the container, and what challenges the whole development and operation mode will face after the introduction of the container. In my opinion, in the early introduction of container technology should be the priority in some non-core running on application development, testing environment, on the one hand, the development and operational colleagues familiar with container scenario development, test, and the whole process based on the virtual machine is different, on the other hand is in the development environment to run tests to verify the container the stability of the platform itself, These two aspects are used to assess when to deploy a production environment and whether to deploy critical systems to a container environment. At the same time, for most application scenarios, in fact, there is little difference between the container running on the virtual machine and the physical machine, but the container running on the host machine had better have a separate network segment, which is conducive to the configuration of network policies and the rapid positioning of problems.
The third is to go to IOE. This is an age-old question, not only from a technical point of view, but also from a political point of view that will speed up the process of de-IoE as the US increasingly confronts China in the field of science and technology as relations between China and the US deteriorate. According to the author's understanding, in IOE, I (IBM) is relatively fast and easy for everyone to go. After all, the foundation of cloud is x86. The architect needs to evaluate the business capacity, importance degree and associated impact, plan the scheme, and design the migration scheme in batches. Now let's look at O (Oracle), which needs to be looked at by industry. At present, most Internet companies in China use MySQL, and the effort to remove O is the biggest. However, in the traditional industry, some core systems still use Oracle. It mainly involves high availability scheme based on MYSQL, double live scheme, library and table separation, read and write separation, etc. Finally, E (centralized storage represented by EMC), in this part, with the popularization of x86-based distributed storage technology, as an architect, we need to consider the feasibility of using distributed storage and the implementation path. The author still suggests using the universal formula: "Develop test environment first" plus "time verification" and then gradually spread to production environment, and eventually replace centralized storage.
Fourth, the introduction of open source software. With the increasing use of Internet technology in traditional industries, a large number of open source software of various kinds have been introduced. This leads to a problem, if the architect is unable to select from the many open source software suitable for their own company, but let the development team to introduce and use their own, it will be a huge disaster. On the one hand, the introduction of multiple open source software of the same type leads to the increase of system docking cost and the decrease of code reuse rate. On the other hand, every time a new open source software is introduced, it needs to customize its specifications, usage standards and security requirements. Moreover, open source software versions are upgraded quickly, with many security loopholes and high maintenance costs. Therefore, as an architect, it is very important to set up a list of options for open source software that are specific to the company, preferably one from each direction (Java container, message-oriented middleware, cache, coordination center, etc.) as a standard option, and if new middleware needs to be introduced, it needs to be evaluated before it is introduced.
The fifth is the elastic expansion of the application. The flexibility of application expansion is also a topic that has been discussed a lot in recent years. It is often discussed together with the second killing of Taobao on November 11. Elastic expansion scenarios are also used in traditional industries, although they may not reach the scale of Singles' Day. For example, now many companies with e-commerce departments will also carry out seckill or promotion. Elastic expansion of application in seckill or promotion scenarios is also a very important part in this scenario. To do elastic capacity expansion, the preferred solution is to transform the application architecture into stateless application, which is also the mainstream practice in the industry. Only after the unified and centralized management of the application state can the application capacity expansion proceed smoothly. However, a practical situation is that it is difficult to do stateless transformation for some applications in traditional industries, so it is necessary to consider making a synchronous action of state after expanding the application capacity before providing external services. This process will take much longer than stateless applications, so the adaptability of some scenarios will be poor. It is also a challenge for architects to be able to design flexible scaling solutions for different applications based on the actual situation of the company.
The sixth is micro-service design. Microservices are not a new concept, but they have become "web celebrity" in recent years as container popularity has increased. It is not exaggerating to say that if you go out to do technology sharing now, if you say that your company has not carried out the construction of micro service, then someone will really think that your company is low. As an architect in the services, you need to know what kind of system is suitable for the service innovation, how to use the domain model design for the application, need to know how to carry on the management of service, the service gateway should be designed in what position, how to avoid carefully after micro service split into micro service "hell", What is the relationship between the microservice and the container?
Seventh, integration of research and development. This involves requirements management, requirements development, testing and deployment online. In the traditional industry, each link basically has a department wall and an independent platform to work on. Therefore, in order to accelerate the launching speed of business requirements and make the business respond to market demands more quickly, it is necessary for an architect to get through all the links mentioned above and build an integrated R&D platform at the company level. In this case, it's not just the technical skills of the architect, but also communication skills, interpersonal skills, and even the ability to manage up.
Next, let's talk about some of the challenges that development positions face.
One is agile development. Due to the long response period, the traditional waterfall development mode has seriously hampered the development of the business and failed to keep up with the changes in the market. The agile development model is about quick iterations and small steps, which requires developers to have a clear understanding of the larger context of the business requirements, so that there is no misunderstanding or additional communication during each small iteration. In addition, in agile development, each iteration has a clear time point and a list of requirements, which has certain requirements on the ability of the developer, will improve the development efficiency of the developer, and to some extent, reduce the time of "paddle".
The second is to adapt to the cloud native development mode. According to the author's understanding, at present, there are not many real cloud native applications in the traditional industry, but the cloud native applications will be the inevitable development path after the landing of cloud computing. Therefore, from the perspective of taking precautions, it is also very necessary for developers to understand the development mode of cloud native applications. And some of the development ideas in the cloud native application can also guide and refer to the existing work.
The third is to adapt to the DevOps model. A lot of people think of DevOps as a tool or a job, but in my opinion, DevOps is a way of working, a way of connecting business people, developers, testers, operations people. Developers are the link between the preceding and the following, and how developers understand and practice the CI phase of DevOps has a big impact on how the whole pattern works. Therefore, in the process of DevOps, the design of CI stage is often based on the actual situation and R&D capability of each company. In other words, the design of CI in a company's DevOps process can also reflect the strength of the company's DevOps capability to some extent.
Next, let's talk about some of the challenges that operations and maintenance positions face.
One is the transformation of basic resources from centralized to cloud. Traditional centralized basic resources focus on the performance and stability of the individual, and the way of thinking about the problem is vertical. But based on the distributed computing infrastructure, operations staff need to consider is the fault of the cloud resource pool how to divide the domain, how to design the size of the cloud resource pool, how different environmental resource pool to isolation, overbooked than what is set to resources, standardization of how to design, cloud resources pool after different range of failure, How to ensure the continued provision of reliable services, these are the cloud after the operation and maintenance personnel need to consider.
Second, operation and maintenance ahead. The author found that the operations and maintenance staff of some companies do things too low-level, only responsible for the OS layer and the following parts, and do not understand the operation and maintenance of the application layer at all, so that when there are problems in the application, it seems that the development is troubleshooting problems, which will give the leaders a bad feeling. At the same time, with the popularity of cloud computing, the content below OS layer is likely to be directly hosted by cloud service providers in the future, just like the hosted room used by many companies now, and many start-up companies directly use public cloud. In extreme cases, the operation and maintenance personnel may be laid off. Therefore, operations personnel should put their skills in front of them. For example, for a company with a Java technology stack, it is better for operations personnel to learn relevant knowledge and skills of Java virtual machines, so that when problems occur in Java programs, they can quickly locate problems, rather than relying on developers.
Third, container operation and maintenance. Container platform after used by a lot of, is bound to cause operational problems, but the container platform of operational mode and operational mode based on virtual machine is completely different, which requires operations staff to familiar with container platform operation mechanism, the container has a theoretical knowledge reserves and practical operational ability, and the skills of traditional operations staff basic is a brand new field.
Fourth, the operation and maintenance of open source software. As mentioned above, the architect needs to conduct professional evaluation and control of the introduction of open source software, so the operation and maintenance of the introduced open source software falls to the operation and maintenance personnel. Traditional corporate operations often buy outside operations services, most of which are for traditional commercial software rather than open source software. Therefore, after the introduction of open source software, the operation and maintenance personnel have no external services to rely on, so they need to have the ability to operate and maintain open source software. The learning cost here is quite high, and there is no so-called "master" standard.
Having talked so much about the roles of architect, development, and operations, what are some ways to help people in traditional industries make the transition? In my opinion, there are the following aspects:
First, independent learning. To quote an old saying "IT is never too old to learn", especially in the age of IT, which is a new and ever-changing technology, self-driven learning is a basic requirement.
The second is to join a group of people who are really talking about and improving technology. At present, there are many social networking platforms and software, but they are mixed. Either some groups use the pretext of technical communication to promote advertising in the group, or no one talks at all after joining the group, or there are all some followers in the group, so it is difficult to find a reliable circle. The better way is through the introduction of "Daniel" or can join the TWT community platform as a representative to carry out from the basic to the advanced training exchange, answer questions;
Third, I actively participate in work by taking advantage of project construction or other opportunities to get in touch with new technologies. Through project construction, I learn the knowledge involved and have a good practice of the knowledge involved with the implementation of the project.