Closed-system assumptions and model limitations | Year 12

01 / LEARN

Understand the idea.

Consistency is not the same as accuracy

A closed system keeps all members within the listed states, so column sums of 1 preserve the total. Repeated powers also assume unchanged transition behaviour over equal periods. Entry, exit or changing categories may require extra states or explicit flows.

Internal consistency does not establish forecast accuracy. Estimated proportions depend on the sample, time period and state definitions. A valid transition matrix predicts expected distributions; real observations can differ even when its columns correctly sum to 1.

Use fresh evidence to evaluate a forecast

Compare forecasts with fresh observations, including errors in each state and in the total. Opposite errors can cancel in a signed total while hiding a large redistribution. Choose an error measure matching the question and retain precision until reporting.

Investigate repeated error patterns and changes in context before revising rates. A single good fit does not establish lasting accuracy. Distinguish a population entering or leaving from a redistribution among existing states, and identify the data needed to explain the difference.

02 / WORKED EXAMPLES

See the method in action.

EXAMPLE 01

A total reveals an omitted process

A closed model predicts (60,40), but observed counts are (58,47). Identify the first warning.

Worked solution

1
Compare predicted and observed totals.
$\begin{aligned} Predicted total & = 60 + 40 = 100 \\ Observed total & = 58 + 47 = 105 \end{aligned}$
The closed model and observation do not contain the same number of members.
2
Quantify the discrepancy.
$105 - 100 = 5$
Transfers between the listed categories cannot add five members to the total.
3
Identify the assumption needing investigation.
Check entry, exit, counting and state definitions before changing transfer percentages; those percentages alone cannot explain a changed total in a closed model.

Answer

The closed-system assumption needs checking.

EXAMPLE 02

Compare two forecasts

Observed counts are (72,28). P predicts (70,30); Q predicts (60,40). Compare maximum absolute state error.

Worked solution

1
Calculate P’s absolute state errors.
$\begin{aligned} ∣ 70 - 72 ∣ & = 2 \\ ∣ 30 - 28 ∣ & = 2 \\ E_{P} & = max {2, 2} = 2 \end{aligned}$
Maximum absolute state error measures the largest miss among the category counts.
2
Calculate Q’s absolute state errors.
$\begin{aligned} ∣ 60 - 72 ∣ & = 12 \\ ∣ 40 - 28 ∣ & = 12 \\ E_{Q} & = max {12, 12} = 12 \end{aligned}$
Use the same observed counts and error measure for a fair comparison.
3
Compare and limit the conclusion.
$E_{P} = 2 < 12 = E_{Q}$
P fits this observation better by the stated measure. More independent observations and assumption checks are needed to judge future forecasting performance.

Answer

Maximum error: P 2, Q 12.

03 / YOUR TURN

Try it yourself.

Check your understanding

Observed totals rise from 500 to 540. Can a closed transition alone explain it?

Show answer & reasoning

No.

No. A closed column-stochastic transition preserves the total, whereas $540 - 500 = 40$ additional members are observed. Entry, exit, counting or category definitions must be investigated.

Keep practising

6 worksheet questions · 18 marks

Open worksheet Worked solutions

Read the questions online

Name two assumptions behind a six-month forecast $T^{6} x_{0}$ .
3 marks
A matrix has columns totalling 1 but an entry $- 0.1$ . Why is it invalid as a transition-probability matrix?
3 marks
Predicted counts are $(120, 80)$ , observed $(112, 88)$ . Find absolute errors and their sum.
3 marks
The same 30 travellers estimate a matrix and are then used to claim its forecast is perfect. Explain a weakness.
3 marks
A model predicts proportions $(0.35, 0.65)$ for 1000 members.
3 marks
1. (a)
  Find expected counts.
  1 mark
2. (b)
  Does observing $(340, 660)$ alone prove failure? Explain.
  2 marks
A new bus route changes behaviour halfway through a year. How should the model respond?
3 marks

Read the worked solutions

Equivalent correct methods earn the corresponding marks. Include units where asked.

Name two assumptions behind a six-month forecast $T^{6} x_{0}$ .
3 marks
The periods must also be equal and state labels consistent.
A constant monthly transition rule and a closed set of states/population.
A matrix has columns totalling 1 but an entry $- 0.1$ . Why is it invalid as a transition-probability matrix?
3 marks
Each entry must be between 0 and 1.
A probability cannot be negative.
Predicted counts are $(120, 80)$ , observed $(112, 88)$ . Find absolute errors and their sum.
3 marks
Signed errors are $(- 8, 8)$ .
Take absolute values before adding.
$(8, 8)$ , sum 16.
The same 30 travellers estimate a matrix and are then used to claim its forecast is perfect. Explain a weakness.
3 marks
Use later or separate observations; the small original sample also limits precision.
The claimed check is not independent of the fitting data.
A model predicts proportions $(0.35, 0.65)$ for 1000 members.
3 marks
a) Multiply each proportion by 1000.
b) One difference of 10 in each state does not establish a changed underlying rule.
1. (a)
  Find expected counts.
  1 mark
  Multiply each proportion by 1000.
  $(350, 650)$ .
2. (b)
  Does observing $(340, 660)$ alone prove failure? Explain.
  2 marks
  One difference of 10 in each state does not establish a changed underlying rule.
  No; assess variation, sample uncertainty and repeated prediction errors.
a) $(350, 650)$ .; b) No; assess variation, sample uncertainty and repeated prediction errors.
A new bus route changes behaviour halfway through a year. How should the model respond?
3 marks
Keep compatible state labels and check whether the population remains closed.
Estimate a new transition rule from suitable later data and apply old/new matrices in the correct chronological order.

Checking saved progress…

Curriculum reference & lesson scope

SACE Board of South Australia

Stage 2 General Mathematics matrix-models option. This is the offered fifth-topic option, alongside the four compulsory topics; a school-designed Open Topic is an alternative, not prescribed content in this chapter.

Official curriculum source ↗

In this lesson

Understand the idea.

Consistency is not the same as accuracy

Use fresh evidence to evaluate a forecast

See the method in action.

A total reveals an omitted process

Compare predicted and observed totals.

Quantify the discrepancy.

Identify the assumption needing investigation.

Compare two forecasts

Calculate P’s absolute state errors.

Calculate Q’s absolute state errors.

Compare and limit the conclusion.

Try it yourself.

Check your understanding

Keep practising