Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: zz-x2580
Data Processing Individual Project
Introduc)on
Things like Python scripts and spreadsheets (such as Microso? Excel) are invaluable in design. They
are free-form. calculaGon tools that allow di?erent design opGons to be rapidly evaluated, tedious
hand calculaGons to be automated, large amounts of data to be manipulated, update calculaGons
and reports as data changes, and veri?caGon. The versaGlity and complexity of these tools, together with how people use them, however, means that they can be very di?cult to follow by someone else reviewing them or even the authors themselves (try opening a spreadsheet or a computaGonal
notebook you wrote a while back, can you follow easily what happened?).
The aims for this self-led learning assessment are:
a) to build upon the knowledge developed in Programming Skills for Engineers 2 by using Python and its ecosystem of libraries to handle, manipulate and process data for typical engineering calculaGons;
b) developing a criGcal understanding of how to process data and check for errors, and communicate about the algorithms about these processes; and
c) to present the calculaGons and their outputs clearly and concisely, so that they can be easily understood, reviewed, and used by someone who has not been involved in the project
before, and in enough detail to give a complete record of the calculaGons, so that it could be built upon by someone else later.
This project is worth 15% of the total TED2 course marks. There is only one, summaGve submission, and the deadline will be communicated by ETO, and it will be via Learn. This is a compulsory
submission, and it will be assessed using the criteria shown inTable 1at the end of the brief. You should submit a single ZIP ?le like the one you were given as an example. The ZIP ?le should
contain all the informaGon needed to follow your calculaGons and produce your reports. The essenGal informaGon will be:
(1) the Jupyter notebook (.ipynb ?le),
(2) the Jupyter notebook exported to HTML (.html ?le),
(3) the report in the Excel spreadsheet (?nal .xlsx ?le), and
(4) any auxiliary ?les that your soluGon may require. These include (a) the raw data you were given, but you may have addiGonal ?les, like (b) the empty template for the excel
spreadsheet, and/or (c) external diagrams that are referenced within the Jupyter notebook (see the example on Learn).
This is a self-led acGvity, and our advice is to try to engage early on with this project and to have a go at things by yourself ?rst. Note the support available to you on this:
1. Videos and example ?les available on Learn.
2. Discussion boards. We expect everyone to give each other support. Sta? monitoring them will typically reply to unanswered posts aNer 2 working days. The link to do so is on Learn.
The project
A new shipping channel is required due to the construcGon of a bridge connecGng Lachtenstein and Crowrovia. The problem is shown graphically inFigure 1andFigure 2. A depth survey has been
conducted of the region, and the survey data for the exisGng seabed pro?le is provide in a raw format in the spreadsheet ?le available on Learn.
You need to calculate the volume of material to be dredged to form the new shipping channel based upon the survey data. An accurate esGmate of volume dredged in m3 is required to esGmate the
costs of dredging the shipping channel. Note you do not need to esGmate the cost, just the volume.
Note that:
1. The calculaGon is not trivial and there are many ways of going about it. It is o?en a good idea to start by solving a simpli?ed version of the problem with pencil and paper before moving
on to the real, more complex, data. It is also a good idea to solve the problem in two or more ways to see if results sGll make sense and are robust to any methodological assumpGons you may have introduced in your approach.
2. The survey data comes in a standard format that holds data that is messy and di?cult to
read by a person. This is very common with large datasets handled by computers. You may assume that:
a. the survey is always carried out in the same way, meaning that the ship surveys a rectangular domain of the seabed following arecGlinear gridwhere the spacing between lines are always the same;
b. the ?le format of the survey data does not change, meaning that its structure will
always be the same for the ‘metadata’ sheet, and that the table in ‘data’ will always have the same number of columns and units, although the number of rows may vary.
3. Take care to understand what units and datums are being used in this survey. This is again a
common occurrence in pracGce and can cause problems (see for example theMars Orbiter Mishap InvesGgaGon Board Phase I Report).
4. The data contains errors due to mis-readings and logging issues during the survey. Your approach and/or calculaGons should be resilient to the worst of these errors.
The intenGon is that in a few years’ Gme a similar survey will take place (a?er the iniGal dredging) to assess if the channel needs to be re-dredged to ensure the required depth is maintained. Your
company want to set out your calculaGons so that this can be done with the minimal of e?ort in the future for this and other projects.
Presen)ng the project
It is very important to take care when construcGng any set of design calculaGons so that somebody else can easily understand it. It must be possible for another engineer to understand, check, and if necessary, extend or modify your calculaGons. Remember that this engineer may not be able to ask you about the content of your ?les so the methods and where the data is coming from needs to be extremely clear to a future engineer.
Making the calculaGons is only part of the current exercise; you need to produce reports in a neatly formaged Jupyter Notebook and Excel spreadsheet – the raGonale for the lager is that colleagues who are not familiar with Python can sGll review the project for themselves. (Note: that the output data is more than just the ?nal volume to be dredged, but short report on the key input data and process as well).
Figure 1: Plan of proposed shipping channel under the new Crowlach crossing. Coordinates in metres from the indicated datum. Not to scale. SecGon A-A drawn in Figure 2.
Figure 2: Typical secGon A-A through the planned Crowlach Straits shipping channel. Lowest astronomical Gde (LAT) data taken from hydrographic o?ce publicaGons.
As requested, here there are hints to help you in your project:
1. Read this brief carefully. Note that the submission includes two reports: one in a Jupyter notebook and one in the spreadsheet. They are both reports about the project, but they should be di?erent as they are meant for di?erent types of audience.
2. Engage with the project early on and seek support if you need it. In our experience, even if you started from scratch in programming, there is plenty of Gme and support to achieve marks over 70% in this project if you engage with it right from the start.
3. Please submit the correct ?les (not RAR or anything else). Check that the ZIP ?le is created correctly and that its contents are as you expect them to be. Note that when doing things correctly the ‘.ipynb’ ?le and the ‘.html’ ?le should have idenGcal content.
4. In a good Jupyter notebook, it is possible to clear all contents and rerun the whole notebook to produce the exact same results. It is a good idea to double check that one can do that using just the ZIP ?le being submiged (i.e., extract the contents in a random folder in your computer and verify it works as intended).
5. Think carefully about the best way to set out and present the data so that you can verify things are working as intended and that somebody else can easily understand it.
6. Always think about the accuracy of your data and results, and only present to an appropriate number of decimal places and include units (just as with hand calculaGons).
7. Bear in mind that somebody looking at your notebook or spreadsheet in the future is
unlikely to have a copy of this brief. A good set of design calculaGons will always include general informaGon, concise and well explained calculaGons and processes (without needing to look in the cell formulae) and will be carefully formaged.
8. If you choose to use informaGon available in this brief, please cite it appropriately as you
would in any other sekng. The library has resources available on this topic if you need help, like access to the service called ‘Cite Them Right’.
9. The example ?les on Learn based on the video (both Python and Excel spreadsheet)
showcase a work in progress to give you an idea of where to begin. Further work would be needed to bring that to the standards expected for high marks on this assignment.
10. The example notebook contains a basic showcase of the libraries and funcGons that you
might ?nd useful. For example,pd.read_csvis useful to read CSV data and has lots of opGons in the documentaGon (click on the link). For this project I would usepd.read_excelsince the data comes in an Excel spreadsheet. Similarly, think about the kinds of ?gures that would be
most informaGve, which are going to be di?erent to the ones used in the example.
11. Asking for help about code is a tricky business.
1) There is the temptaGon of thinking that a piece of code that executes without coding errors is working as intended. That is not necessarily the case.
2) SomeGmes coding errors reveal mistakes one might be making (like `“a” + 1` fails to add `1` to the string `a`). Some other Gmes they can be used as safeguards for things that should not work (like `1/0`, division by zero) or that your algorithm does not
support (like raising an error if the data does not saGsfy key assumpGons).
3) Programming languages have mechanisms to report coding errors. Seeone of the many tutorialsavailable on how to read them for the Python language. Once you
know what the coding error is about, it is easier to ?nd help online. For example,
`1/0` raises in Python a `ZeroDivisionError` and searching for that error leads to, for example,this page.
4) When asking someone else help with a problem, it is good pracGce to create a Minimal, Reproducible Examplewhenever possible.
