Optimizing Maintenance Planning in the Production Industry Using the Markovian Approach

Maintenance is an essential activity in every manufacturing establishment, as manufacturing effectiveness counts on the functionality of production equipment and machinery in terms of their productivity and operational life. Maintenance cost minimization can be achieved by adopting an appropriate maintenance planning policy. This paper applies the Markovian approach to maintenance planning decision, thereby generating optimal maintenance policy from the identified alternatives over a specified period of time. Markov chains, transition matrices, decision processes, and dynamic programming models were formulated for the decision problem related to maintenance operations of a cable production company. Preventive and corrective maintenance data based on workloads and costs, were collected from the company and utilized in this study. The result showed variability in the choice of optimal maintenance policy that was adopted in the case study. Post optimality analysis of the process buttressed the claim. The proposed approach is promising for solving the maintenance scheduling decision problems of the company.


Introduction
A number of optimization techniques have been applied in solving maintenance planning problems.The mathematical programming techniques are useful in finding the minimum of a function of several variables under a prescribed set of constraints.Stochastic process techniques have been used to analyze problems which are described by a set of random variables with known probability distributions.The statistical methods enable one to analyze maintenance data and obtain, from the result, the most accurate representation of the physical situation.
Researchers have applied optimization techniques to develop a sustainable framework for effective planning and optimization of maintenance systems in production organization.Optimization techniques utilized include: dynamic programming (Ozgur-Unluakin and Bilgic 2006; Ming et al. 2004), simulation based on genetic algorithms (Azadivar and Shu 1998), and failure and reliability models (Sheik et al. 1989;Inegbenebor and Adeniji 2002).The aforementioned techniques were used to plan and optimize maintenance systems separately instead of following a given maintenance policy such as preventive, predictive, or corrective policy.An approach that will be flexible and at the same time provide switching mechanisms among the maintenance policies is required in order to allow searching for alternative maintenance policies that can be carried out at a reduced cost.
Previous studies have identified the Markovian approach as a popular approach which has adequate elements of a switching mechanism.Based on this, Markovian approaches have been used by researchers in studying dynamic change in vegetation types (Debussche et al. 1977), characterizing ecological successions (Usher 1979), forecasting air pollution levels (Anthony and Taylor 1977) estimating land use changes (Vandeveer and Drummond 1978) recognizing speeches (Rabinar 1989) estimating the service life of bridge elements (Ansell et al. 2001) planning for manpower (Elliott and Siu, 2009;Tsao et al. 2009;Forbes and Batholomew 1979) recognizing human activities (Duong et al. 2009) and solving hidden aspects of human endeavours including imaging, tracking and favoritism (Bayraktar and Ludkovski 2009;Aas et al. 1999).
There have been scanty efforts in using the Markovian approach in solving generalized maintenance problems.The approach that is employed in maintenance planning and optimization in this study is a stochastic approach called the Markov decision process (MDP).MDP comprises sets of states, actions, and transition probability matrices that depend on the choice and reward of actions taken within a given state from which future actions are determined.This process is used for forecasting maintenance activities from which an optimal cost-efficient maintenance planning policy, is selected in a given finite horizon.
The rest of the paper is as follows: a review of past work on maintenance planning and the Markovian approach is in Section 2; a formulated model for maintenance planning in the production industry is given in Section 3 and Section 4 presents a plan for model implementation using a cable production company as an example scenario.Results, discussion, and conclusions are respectively detailed in Sections 5 and 6.

Literature Review
A lot of effort has been expended by past researchers in the areas of maintenance planning.Ozgur-Unluakin and Bilgic ( 2006) applied a dynamic probabilistic approach known as dynamic bayesian networks (DBN) to develop a predictive maintenance model for system components with a constant failure rate.This study planned maintenance along the line of predictive policy only.A simulation-optimization procedure based on genetic algorithms was also developed by Azadivar and Shu (1998) for optimization of maintenance policies.The study failed to include a maintenance policy switching mechanism.Sheik et al. (1989) modeled the average rate of occurrence of failures which provides the means of predicting maintenance tasks.In this model, there is a high tendency toward overestimation or underestimation of maintenance tasks, since failure rates are not always constant.A solution methodology based on the stochastic dynamic programming model was developed by Ming et al (2004).In this study, transition probability from one maintenance environment to the other has not been addressed.Inegbenebor and Adeniji (2002) also presented an analysis of the optimal production and corrective maintenance planning problem for failure prone manufacturing systems.Though work-linked production and maintenance activities are generally addressed together, only the solution to static corrective maintenance planning policy problems was addressed.Many of the identified efforts planned and/or optimized for maintenance systems are expressed through a given maintenance policy such as preventive, predictive, or corrective maintenance.There is a need to investigate the possibility of determining an optimal maintenance policy from the identified policies to be adopted in an organization within a given finite time horizon.This study extends the past work to include an optimal choice from amongst different policies that will optimize the economics of maintenance of machinery/equipment in the manufacturing industry at a given finite horizon using the Markovian approach.
One of the early applications of the Markovian approach was outlined by Debussche et al. (1977) in their study of vegetation.They formulated a model based on mapping of vegetation types, and studied dynamic changes among key vegetation species.Usher (1979) suggests that complex non-random or Markovian processes are likely to characterize all ecological successions.The use of Markov models in forecasting air pollution levels was also explored by Anthony and Taylor (1977).Vandeveer and Drummond (1978) applied Markov processes for estimating land use changes, most especially where a major impact is imposed upon an existing system.Rabinar (1989) applied hidden Markov models in the area of speech recognition.Markovian based models have been applied in manpower planning in the manufacturing industries in the areas of optimal portfolio choice (Elliott and Siu 2009), promotion effects on retention rates (Tsao et al. 2009), and human activity recognition (Duong et al. 2009).Practical applications of the Markovian method in manpower planning were also suggested by Forbes and Batholomew (1979).More recently the Markovian approach has been extended to solve hidden aspects of human endeavours including imaging, tracking, and favoritism (Bayraktar and Ludkovski 2009;Aas et al. 1999).Ansell et al. (2001) present a Markov approach in estimating the service life of bridge elements, while riskbased inspection maintenance was also modelled by Corotis et al. (2005).None of the aforementioned studies have applied the Markovian process to maintenance planning except those of Ansell et al. (2001) and Corotis et al. (2005), and those studies are limited to service life estimation and risk-based inspection, respectively.There has been little effort in the area of application of the Markovian approach in solving a generalized maintenance problem where global maintenance policies (preventive, predictive, and corrective) are considered.This study applies the Markovian approach to establish the economic choice of maintenance policy in the production organization, by taking into consideration the stochastic nature of industrial maintenance workloads.

Model Formulation
This comprises a detailed analysis of procedural steps and the approach taken in achieving the end results.Considering preventive, predictive, and corrective maintenance policies, volume of work is categorized based on time spent on the maintenance carried out.In many production industries, such as breweries, beverages manufacturers, and cable production companies, the production process involves a number of stages.Each stage has one or more machines or pieces of equipment that executes operation(s) peculiar to that stage.Whether failure is predicted or not, preventive and corrective maintenance are carried out on these machines whenever required, and cost is incurred directly or indirectly for the maintenance operations that are carried out.Based on this, only preventive and corrective maintenance functions are considered in this study.Also, simultaneous performance of both preventive and corrective maintenance actions on the machines at a given time is discouraged.The parameters considered in formulating the model are presented in the following section.

Time and Cost Analysis
To determine the maintenance workload and corresponding cost of maintenance, time aggregates and incurred maintenance costs are determined.Assuming that the total budgeted preventive maintenance time for all machines is not exceeded, then the average preventive maintenance time t p spent on each machine is given by: (3.1)Similarly, if corrective maintenance action is performed, the average corrective maintenance time t c spent on each machine is given by: (3.2) Therefore, total preventive maintenance time, q p , and corrective maintenance time, q p , spent on each machine per month are respectively expressed by Eqns.3.3 and 3.4: (3.3) (3.4)By substituting Eqn.3.1 into Eqn.3.3, it gives the total preventive maintenance time: (3.5) Similarly, total corrective maintenance time can be calculated as follows: (3.6) Therefore, the total time spent on preventive maintenance of all machines per month, Q p is: For a quarterly estimate (three months, k =1, 2, 3), total time spent on preventive maintenance of all machines, Q p is: Similarly, the total time spent on corrective maintenance of all machines per month, Q c is: In many developing countries such as Nigeria, the maintenance workload is usually preplanned quarterly (that is every three months).Since simultaneous performance of both preventive and corrective maintenance functions is not allowed if preventive maintenance is scheduled for this quarter, any other correc-tive maintenance function will be contracted out (if critical) or delayed to the next quarters (if not critical).Anually, there are four quarters in which either preventive or corrective maintenance can be carried out.Based on the quarterly maintenance workload estimates of past years, the outcomes, transition maintenance costs and workload probability matrices are used to determine future maintenance costs under a preventive, or corrective maintenance policy.The policy and quarter corresponding to minimum maintenance cost is chosen as optimal.The following sections explain how transition chains and matrices are formulated.

Transition Chain
Corrective and preventive maintenance are grouped into four states or quarters, S 1 , S 2 , S 3 , and S 4 , according to maintenance workload, and using time spent on maintenance practice and costs (Eqns.3.7, 3.8, 3.12 and 3.13).On the basics of these equations, transition probability matrices are estimated based on a timebased maintenance workload.The maintenance workload transition process is in Figure 1.

Transition Matrix
A transition probability matrix M y is formed using the transition probabilities of the four quarterly states (Figure 1) as follows where y represents maintenance alternatives: (3.14a) In this equation, y = 1, and 2, is assumed for preventive, and corrective maintenance policies, respectively.Simultaneously, i, j represent transition among quarterly states (S 1 , S 2 , S 3 , S 4 ) of the workload probability, (m ij) (Figure 1).
(3.14b)Each maintenance alternative is accompanied by maintenance costs (or rewards).Matrix C y correspond to m ij , and is presented by Eqn.3.14b:

Decision Process Optimization Model
The appropriate maintenance policy corresponding to minimum maintenance cost is obtained using the Markovian process based on dynamic programming.The problem is to find maintenance policies (y) (y = 1, 2) and state(s), in quarters S 1 , S 2 , S 3 , and S 4 of stage x, that minimize(s) maintenance cost (C y ) over a planned finite time horizon (N).
Let w = number of states, j for each year x = stage or year N = maximum number of stages (years) under consideration y = maintenance option ƒ x (i)= optimal expected maintenance cost of quarter, i, in stage x (Vy)i = expected maintenance cost of quarter, i, given option y C a = annual maintenance budget C q = quarterly maintenance budget f = inflation factor The backward recursive equation relating ƒ x and ƒ x+1 , for determining optimum expected maintenance cost of quarter i in stage x is written as: (3.15) The expected maintenance cost in quarter i of j's at a given option y is expressed as: (3.16) The optimum expected (actual) maintenance cost of stage N is given as: (3.17)By substituting Eqn 3.16 into equation 3.15, it gives: (3.18)For a specified period of N years, Eqns.3.17 and 3.18 give an iterative approach for selecting an appropriate economic maintenance planning policy, ie., optimal policy to `be adopted at minimum cost over the planned time horizon, N).It is pertinent to note that full implementation of a preventive or corrective maintenance policy, or both policies, in all quarters of the year, may not be economical in the long run.Instead, the cost of maintenance can be reduced by balancing the preventive and corrective maintenance activities.The problem is to find which quarter(s) that the preventive and/or corrective maintenance workloads can be scheduled in order to minimize costs.Based on 4 x 4 matrices of the quarterly maintenance programs considered in this study, there are sixteen (16) possible scheduling alternatives.They are: * practice preventive maintenance in the first quarter and corrective maintenance in the second, third, and fourth quarters (PCCC).* practice corrective maintenance in the first quarter, preventive maintenance in the second quarter and corrective maintenance in the third and fourth quarters.(CPCC).* practice corrective maintenance in the first and second quarters, preventive maintenance in the third quarter, and corrective maintenance in the last quarter (CCPC).* practice corrective maintenance in the first three quarters and correctives maintenance in the fourth quarter (CCCP).To verify whether optimal maintenance options obtained at every quarter of each stage can be practiced by a particular company, optimal expected maintenance cost of quarter, i in stage x, f x (i), is further compared with the company's quarterly maintenance budget, C q for a given annual budget of C a , where stage, x (year) =1,2,..,N.determines the future annual budget under inflation, f (%), arrangement.The expected annual maintenance budget under influence of inflation is computed from: The corresponding quarterly estimate (comprises four quarters per year) is given as: The positive value of Eqn.3.21 is an indication of budget surplus, which means that money remains left after maintenance which can be spent on other productive activities.The negative value means that the budgeted money is not enough to complete maintenance activities; hence, supplementary budgeting may be necessary.

Software Development
A computer software package, which carefully executes the decision process, was developed in Microsoft Visual Basic (VB Redmond, Washington, USA, Version 6.0) programming language.Computer programming was carried out for easy implementation of the model in the industry, where an analytical approach could lead to waste of man-hours.Spreadsheets software such as Microsoft Excel template can also be used (Taha 2008).Excel software, apart from cost, requires special training and may be cumbersome for maintenance personnel.The computer software developed using VB is interactive, userfriendly and easy to use by ordinary people.The algorithms developed for the program are as follows: 1 Start 2 Input array dimension, w 3 Input values for preventive maintenance probability matrix (m 1 ) ij 4 Input values for preventive maintenance cost matrix (C 1 ) ij 5 Input values for corrective maintenance probability matrix (m 2 ) ij 6 Input values for corrective maintenance cost matrix (C 2 ) ij 7 Specify the number of stages/years of maintenance for consideration, N 8 Initialize y to 1 9 While y is = 2 begin 10 Initialize i to 1 11 While i < = w, begin 12 Let (V y )itemp = 0 13 Initialize j to 1 14 While j < = w, begin Table 1.Monthly maintenance time analysis for the year 2005 maintenance activities.By using these data, maintenance workloads, total time spent on maintenance, and minutes, (q p , q c ) for each machine per month are computed using Eqns.3.5, and 3.6, respectively, for preventive and corrective arrangement, while Eqns.3.7a and 3.8a are used to estimate monthly maintenance workloads on all the machines (Q p , Q c ).The corresponding monthly costs for carrying out the preventive and corrective maintenance policies (C p , C c ) are computed using Eqns 3.12a and 3.13a, respectively, and the results are presented in Table 2.The same procedure was used to analyze maintenance data for the year 2006.The resulting maintenance workloads and costs from the analysis (year 2006) are shown in Tables 3  and 4, respectively.In all cases, the maintenance data/outcomes are tabulated with each sub-heading according to the parameters denoted in the nomenclature given in the previous section.
The monthly outcome data obtained for the both preventive and corrective maintenance for years 2005 and 2006 were used to compute quarterly workload (Q p ', Q c ') and cost (C p ', C c ') using Eqns 3.7b, 3.8b, 3.12b and 3.13b, respectively.The quarterly outcome data are further analyzed via the developed Markovian-based model (Eqns.3.14-3.21)using analytical and computer software methods.The results obtained from the analyses are presented and discussed in the following section.

Results and Discussion
In order to obtain both preventive and corrective transition probability matrices, and their corresponding cost matrices, maintenance data obtained are processed as shown in this section.

Preventive Maintenance
Table 5 shows the quarterly maintenance workload (in minutes) obtained for preventive maintenance, Q p ' in the years 2005 and 2006, respectively, for the cable company.The transition probability of the maintenance workload from 2005 to 2006 is determined from the ratio of the absolute difference between workloads of respective quarters of years 2005 and 2006 to the total workload for the years.The summation of the values in each column is given in the last row.Each of the quarterly transition workload results is divided by the total sum in the last row.This gives a total probability of one for each column, as presented in Table 6.
Therefore, preventive maintenance (y=1) workload transition probability matrix M 1 is written as follows: Quarterly preventive maintenance cost, C p ', are presented in Table 7. Transition cost elements are obtained by taking the average of the quarterly cost transitions from 2005.
Preventive maintenance cost matrix is written as:

Corrective Maintenance
The quarterly transition probability and cost matrices for corrective maintenance between the years 2005 and 2006 of the company are estimated using methods similar to that of the preventive maintenance policy.The quarterly workload results (in minutes) Q c ' are presented in Table 8, while its corresponding probabilities are in Table 9.
Corrective maintenance (y = 2) probability matrix, M 2 , is written as: Table 10 presents the corresponding quarterly corrective maintenance cost, C c ' associated with each transition probability.This is obtained in similar way as preventive policy.
Thus, the corrective maintenance cost matrix is: For a 10-year maintenance plan (N = 10), using analytical and software methods, the outcome matrices were input into the developed Markovian based dynamic programming model, the optimal policy results obtained are presented in Table 11.The paired t-test results of the comparison of the maintenance cost generated by software and manual mathematical com-Table 4. Monthly maintenance cost foer the year 2006 putation show that there is no significant difference between them (that calculated value of t, 4.09 x 10-6, is far less than the table value of t, at 2.704).This means that the developed software can adequately replace the computation using analytical method, which is somewhat tedious to accomplish.Through software calculations, the computations was accomplished within 2 minutes, while it took several hours to get results from manual calculation.
The result in Table 11 shows that the optimal maintenance planning policy would be one adopted over a period of ten years where preventive maintenance is practiced in the first and third quarters of every year, while corrective maintenance is practiced in the second and fourth quarters of every year.This recommendation is based on the fact that the cost of maintenance is minimized in those quarters.
The Company's annual maintenance budget is N5.85 million (N, is the symbol for Nigeria's currency, the naira), while Nigeria's inflation rate is 25% (Akanbi et al. 2001).To further verify the company's capability of adopting this maintenance policy with its available resources, the quarterly maintenance budget for the old method was compared with the quarterly optimal maintenance cost from the current schedule.
The results obtained (Table 12) show that the compa-ny can actually adopt the proposed policy.The results show that out of the sixteen scheduling arrangements, only PCPC (practice preventive maintenance policy in the first and third quarters, and then corrective maintenancne in the second and fourth quarters), is optimal.With this optimal scheduling, a surplus in the maintenance budget would be achieved as shown in the last column of Table 12.This shows that the company's traditional method of maintenance has not been economical.Therefore, the company has been advised to adopt the new PCPC approach to avoid supplementary budgeting for maintenance activities.By carrying out post optimality analysis, a performance ratio was obtained for each maintenance option in every quarter of the year, using the results of the actual quarterly maintenance cost obtained in Table 11 for a period of 10 years (Table 13).Figure 2 displays a graphical illustration of the performance ratios for both preventive and corrective maintenance during a 10-year period.For years 1 and 2, preventive maintenance has a fairly high performance ratio (P r ), which is higher than corrective maintenance during quarters 1 and 3, while the P r of corrective maintenance is higher for quarters 2 and 4.There is a slight difference in the P r of preventive and corrective maintenance during quarters 2 and 3 of year 3, but there is a greater margin in quarter 4 with corrective maintenance having a higher P r .In year 4, the P r is zero for preventive maintenance in quarter 2 and has a negative value in the fourth quarter.The implication of this is that preventive maintenance actions during these quarters has a great cost disadvantage to the organization; instead, corrective maintenance is economical during these quarters.

Table 12. Cost comparison for old and new scheduling approach
The most cost efficient maintenance option in quarter 1 of year 5 is preventive maintenance while that of quarter 4 is corrective.Preventive maintenance in quarter 4 of year 6 has a P r of zero; therefore corrective maintenance stands out as the better option to be taken for quarter 4 of that year.In years 8, 9, and 10 preventive and corrective maintenance exhibit the same P r in quarters 1 and 3, but with disparity in quarter 4 where corrective maintenance has a higher P r .
The illustrations in Figure 2 further support the efficacy of the optimality results presented in Table 11 the cable production company should practice pure preventive maintenance in the first and third quarters and pure corrective maintenance in the second and fourth quarters.This will prevent over-spending in some areas of maintenance set-up.

Conclusions
The Markovian approach applied in the determination of an optimal maintenance planning policy could be useful not only in production industries, but also in industries that provide services to customers.The procedure involves critical analysis of maintenance activities relating to workload and cost.This study provides a good approach to maintenance planning for either Table 13.Performance ratio (P r ) for preventive and corrective maintenance production or service industries which have maintenance departments that handle the maintenance of their equipment and machines.However, its effectiveness can only be appreciated if there has been proper documentation of maintenance work in th past, since data analyses are essential to arriving at an accurate result.
Since it becomes impracticable to completely ignore a particular maintenance alternative over a certain period of time, there is a need to determine optimal maintenance policies to follow over that period.This will solve the problem of overspending on maintenance, especially when eliminating of multiple set-up costs.Other factors, such as, personnel/manpower, utilizastion, and materials and spare part management contribute to the effectiveness of maintenance plan- ning in the production industry.The proposed approach is promising to solving the problem of overbudgeting for maintenance activities in the production industry.Further work on testing the model using an infinite time horizon will likely yield steady state optimal results in which a corrective maintenance policy is likely to be prominent.In this case, a conditional failure probability analysis for the components of the machine is essential in which additional constraints must also be considered.For such a study, the objective could be to establish period(s) when the failure rate of component(s) will minimize the cost of corrective maintenance on the machine.
incidences of preventive maintenance on a machine per month, (a random variable) n c = number of incidences of corrective maintenance on a machine per month, (a random variable) Q p = total hours spent on preventive maintenance of machines per month Q c = total time spent on corrective maintenance of machines per month C s = spare parts and material costs per month C u = utility costs per month C l = labour costs per month ______________________________________________ *Corresponding author's e-mail: karbil2002@yahoo.com For a quarterly estimate (three consecutive months, k = 1, 2, 3), Q c is: (3.8b) Spare parts cost C s per month and calculation by: (3.9)where i = 1,2,…,s as a counter for number of spare parts.Labour cost per month (C l) is given as: (3.10)where 1 = 1,2,…,l as a counter for the amount of labour.Utility cost per month (C u ) is given as: (3.11)where i = 1,2,…,u, as a counter for the number of utilities.Maintenance cost per month (C p ) for preventive maintenance policy is given as: (3.12a)The corresponding quarterly estimate, C p , is (3.12b)Maintenance cost, per month (C c ) for corrective maintenance policy is given as: (3.13a)The corresponding quarterly estimate, C c is (3.13b)
) x /4 (3.19b)By carrying out post optimality analysis, perform-ance ratio (P r ) is determined for each maintenance option in every quarter of each year, based on the actual quarterly maintenance cost: P r = actual cost/ budget (3.20)Deviation from the maintenance budget is obtained by finding the difference between the budgeted and actual cost of maintenance.Deviation = Budget -Actual cost (3.21)

Table 2 .
Monthly maintenance cost for the year 2005

Table 3 .
Monthly maintenance time analysis for the year 2006

Table 5 .
Quarterly preventive maintenance workload analysis

Table 6 .
Quarterly preventive workload transition probabilities

Table 7 .
Quarterly preventive maintenance cost transition

Table 9 .
Corrective maintenance workload transition probabilities

Table 10 .
Quarterly corrective maintenance cost transition

Table 8 .
Quarterly Corrective Maintenance Workload Analysis

Table 11 .
Optimal maintenance policy results