Hello, dear friend, you can consult us at any time if you have any questions, add  WeChat:  zz-x2580
ATHK1001 ANALYTIC THINKING: ASSIGNMENT 

Online submission: All submissions are to be made online on the ATHK1001 Canvas website.
Submissions will be checked for plagiarism. Artificial Intelligence tools such as ChatGPT that assist
writing are not permitted.

Incorrect submissions: If you discover before the closing date that the file you submitted on
Turnitin was incorrect, and let us know, you may be given the option to resubmit a corrected
version with a 50% penalty or the relevant lateness penalty, whichever is greater.

Word length: 750 words across all questions (excluding references in Question 13). A penalty of
10% will apply to papers that exceed this limit by more than 10%, a 20% penalty if you exceed
20% of the limit, and 30% if you exceed the limit by 30%.

Total marks: 60 (15% of total grade for class)

Background and Aims
A useful skill when dealing with data is being able to estimate plausible answers to questions you
don’t know the answer to. A strategy for doing this is embodied by what are known as “Fermi problems.”
These are numerical estimation problems that break down a difficult estimation problem into steps. A
common example is “How many piano tuners are there in Chicago?” Most people have no idea of what
the right answer may be, but it can be broken down into a set of estimations about which people have
more confidence: How many people are there in Chicago? How many families? What proportion of
families have a piano? How often does a piano have to be tuned? How long does it take to tune a piano?
How many days a year would a piano tuner work? People can come up with reasonable estimates for the
sub-questions and then combine them to make a reasonable estimate for the main questions. They are
called Fermi problems after the Nobel-prize winning physicist Enrico Fermi who reportedly estimated the
force of the first atomic explosion from how far a dropped piece of paper travelled. He was famous for
being able to use such meagre pieces of information to derive surprisingly good answers to questions.
Most famously Fermi problems are the basis for the Drake equation for estimating the number of extra-
terrestrial civilizations. In both science and engineering education Fermi problems are used to show
students the power of deductive thinking, introduce mathematical modelling, and prepare them for
experimental laboratory work. As such they have been used at educational levels ranging from primary to
tertiary (see ?rleb?ck & Albarracín, 2019).
?rleb?ck and Albarracín (2019) conducted a systematic literature review and identified 91 articles
that addressed Fermi problems. Forty-three of these articles are described as empirical studies focused on
teaching or learning using Fermi problems. These have demonstrated that Fermi methods can be taught,
but ?rleb?ck and Albarracín report that although many of the articles advocate for the benefits of
teaching Fermi problems to students none of the reviewed research focused on explicitly establishing
evidence supporting the argument that Fermi problems improve students’ estimation skills. So there
appears to be no published research that addresses the basic question: Does treating a problem as a Fermi
problem lead to better estimates?
The experiment conducted for this assignment addressed the question of whether participants
make more accurate estimates for Fermi problems rather than non-Fermi problems. It does by treating the
same question as a Fermi problem when it is broken down into sub-questions, or a non-Fermi problems
the target question is presented alone. To evaluate this issue, we tested a set of hypotheses using the data
we collected in tutorials in Week 2.

Method
Participants
A total of 197 students from analytic thinking course (ATHK1001) participated as part of a class
experiment were analysed. A larger number participated but were eliminated from the analysis due to
2
their data not being correctly recorded, they completed too little of the experiment, or they did not give
consent to having their data analysed. Of the analysed participants 92 were female, 100 were male, and
they had a mean age 19.3 years.

Materials

All question used were given as either Non-Fermi problems or Fermi problems. Fermi problems
asked the participant to answer a series of four or five sub-questions before answering the main question.
For example, the Fermi problem version of the question “How many piano tuners do you think there are
in Chicago?” first asked:
What do you think is the population of Chicago?
How many pianos do you think there are in Chicago?
How many hours do you think it takes to tune a piano?
How often do you think pianos are tuned over a ten-year period?
How many hours a year do you think the average piano tuner works?
The Non-Fermi problem version just asked “How many piano tuners do you think there are in Chicago?”
Two sets of nine questions each were used. Most Fermi problems presented in lists of Fermi
problems seems to encourage estimation by multiplication sub-question answers, such as appears to be
the case for the Chicago piano tuners question. However, some questions seem to encourage addition of
sub-question answers. For example, the question “What is the total military budgets of the members of
the UN Security council (USA, Russia, China, UK, France), in millions of $US?” was broken down into
the sub-questions:
What is the military budget of the USA, in millions of $US?
What is the military budget of Russia, in millions of $US?
What is the military budget of China, in millions of $US?
What is the military budget of the UK, in millions of $US?
What is the military budget of France, in millions of $US?
So, we constructed a set of nine multiplicative questions and a set of nine additive questions. A list of the
nine questions for each set together with their sub-questions can be found in the Appendix. Every
participant received both sets of questions: one set as Fermi problems (i.e., with sub-questions) and the
other set as Non-Fermi problems. The experiment had two conditions: in the “Multiplicative condition”
participants answered the nine multiplicative questions as Fermi problems and the nine additive questions
as Non-Fermi problems, where as in the “Additive condition” participants answered the nine additive
questions as Fermi problems and the nine multiplicative questions as Non-Fermi problems.

Procedure

During tutorials for the class Analytic Thinking at the University of Sydney participants
completed the experiment individually on computers in the classroom or online. All participants were
randomly assigned by the computer to either “Multiplicative condition” or the “Additive condition” and
then they answered either their nine Fermi problems then their nine Non-Fermi problems, their nine Non-
Fermi problems then their nine Fermi problems. The order in which they answered the problems was
varied randomly. Responses to each question or sub-questions were entered into textboxes on a webpage,
and participants had to enter numbers in response to every question before they cold advance to the next
page.
After completing the experiment participants indicated whether or not they consented to having
their data included in the data set.

Hypotheses

We proposed four hypotheses to investigate how participants performed on Fermi and Non-Fermi
problems.

3
First, we will test whether Fermi and Non-Fermi problems differed in terms of accuracy, and based on
the wide endorsement of Fermi problems we predicted that participants would do better on Fermi
problems.
Hypothesis 1: Mean accuracy for Fermi problems will be different than for Non-Fermi problems.
We predict accuracy will be greater for Fermi problems.

Hypotheses 2 and 3 tested if which set of questions were answered as Fermi problems would make a
difference. We had no reason to predict a difference.
Hypothesis 2: Mean accuracy for Fermi problems will not be differ between “Multiplicative
condition” and the “Additive condition”.

Hypothesis 3: Mean accuracy for Non-Fermi problems will not be differ between “Multiplicative
condition” and the “Additive condition”.

Hypothesis tested whether how accurate participants were on Fermi problems was associated with their
accuracy on Non-Fermi problems.
Hypothesis 4: Accuracy for Fermi problems will correlate with accuracy for Non-Fermi problems.


Results

The data set for our class can be found on the Canvas site for ATHK1001 under “Assignment 1”.
This assignment description can be found there as well as an Excel file called “Assignment1_dataset.xls”.
This Excel file contains all the data for the assignment and has 197 data lines, one for each participant.
Each participant has values for four variables, and the values of each variable are in a single column of
the file.
The first variable is an arbitrary id number generated by the computer. The “condition” column
gives each participant’s condition: “1” means participants were in the “Multiplicative condition” and “2”
means they were in the “Additive condition”. The third variable is the participants accuracy on their
Fermi problems. The third variable is the participants accuracy on their Non-Fermi problems.
Accuracy was calculated based on whether participants’ answers were the same order of
magnitude as the correct answer. The order of magnitude of an answer is an approximation of the
logarithm (base 10) of a value, and can be understood as the numbers of digits in a integer answer minus
1. For, example, any answer between 100 and 999 would have and order of magnitude of 2, because 100
is 102. In estimation which is expected to be inaccurate often the aim is to estimate to the correct order of
magnitude, and sometimes this is explicitly seen as the goal when setting a question up as a Fermi
problems. The Appendix presents the correct order of magnitude for each question.
For a participant’s Fermi problems how many of their answers had the correct order of magnitude
was countered up and then divided by the number of questions they answered, so the Fermi accuracy was
a proportion between 0.0 and 1.0. Similarly, participants’ Non-Fermi accuracy was the proportion of their
Non-Fermi problems that their answers had the correct order of magnitude.