At the beginning of each week, a machine is in one of four conditions (states)
Machine ReplacementAt the beginning of each week, a machine is in one of four conditions (states): excellent (E), good (G), average (A), or bad (B). The weekly revenue earned by a machine in each type of condition is as follows: excellent, $100; good, $80; average, $50; bad, $10. After observing the condition of a machine at the beginning of the week, we have the option of instantaneously replacing it with an excellent machine, which costs $200. The quality of a machine deteriorates over time, as shown in Table.Next Period’s States of MachinesPresent StateProbability That Machine Begins Next Week Asof MachineExcellentGoodAverageBadExcellent.7.3——Good—.7.3—Average——.6.4Bad———1.0until replacedFor this situation, determine the state space, decision sets, transition probabilities, and expected rewards.