Skip to main content Skip to navigation
Hydrogen Properties for Energy Research (HYPER) Lab ME 406

ME 406 Lesson 7: Visualizing your Results

We’re now to the point where we’ve taken measurements and analyzed our confidence and uncertainties. One of the most rewarding parts of experimental investigations is graphing and visualizing your hard work. Usually this will appear in the Results and Conclusions section of your report.

5. Results and Conclusions

What would someone take away from your report if they only read the introduction and skimmed ahead to the results and conclusions? Make sure you don’t let them miss your most important points and findings! When you skim through something what do you look for? Odds are headings and images.

Why the images?

They’re worth a thousand words… 900 of which are irrelevant. But what do they really tell us?

Comparing with visuals

When you look at the above graph, why have I chosen two y-axes to present both the heat capacities of hydrogen and the equilibrium fraction of spinning vs. non-spinning hydrogen at cryogenic temperatures? Because they are related. Graphs are great at showing how sometimes complex information is related, and changes, relative to another. Graphs compare and contrast.

Many will say that figures are a matter of taste and preference. That’s true to a point. However, you’ll find that good graphics will consistently do similar things. Here’s a few examples:

  1. What not to do: University of Wisconsin-Madison Professor Rod Lake’s abominable graphs website.
  2. What to do: here’s a very old standard on graphic design from 1914, but is still very relevant.

In general, the guidelines are pretty clear:

  1. Be empathetic to your reader.
  2. Focus on the relevant change or correlation you want the graphic to display.
  3. Give the titles, units, and digits when and where they are needed.
  4. Omit useless and repetitive words, digits, and space.

But you’ll need to practice to get good at it. The example above was of theoretical curves. Notice that they are smooth. The uncertainties in the predictions likely fall within the line-width. Experimental measurements are seldom this easy. The following figure shows some raw measurement traces for when I measured the visco-plastic flow of solid deuterium. You can see in the figures when steady state was achieved at a given temperature, dynamic shear strength, and heat transfer measurement. At these steady state points I can average to obtain representative points.

Raw experimental measurements

Notice that I use the figure caption to describe what is displayed and do not repeat the information with a title above the graph. These graphs are all made with EES. The EES default graph is very clean and tends to work in most situations. Even for very large datasets like the one above. Plotting raw data like this can give people incredible confidence in your experiment. Can you estimate the precision error from the data traces? Does it look like I achieved steady state before moving to another data point?

With these raw datasets it’s straightforward to average the values to return points you want to report. Here’s an example with all of the points I measured for my Ph.D. dissertation.

Comparison plot

This single graph contains a considerable amount of information. This is a graph you want to close with. This one contains the visco-plastic flow measurements for hydrogen, deuterium, and neon. Can you compare and contrast? I even include the prior measurements, some with estimated uncertainty bars. Do the bars add important information when comparing to the correlations I developed (colored lines)?

In the end, spend time on effective visuals. The narrative that explains them is only necessary when your graphs are not completely obvious alone. Once finished, ask yourself what a person will take away from it. Is it what you want them to leave the report with? It should be. Make sure they can’t miss it.

From here, it’s easy to create bulleted lists summarizing the key findings from your report. With all of the key portions of your reports and presentations covered, it’s time to revise, polish, and practice!

ME 406 Lesson 6: Data Analysis and Uncertainty Propogation

Now that we’ve covered the design/layout, procedure, and instrument calibration and traceability for our experiment, it’s time to start analyzing our data.

Section 4.3: Data Analysis and Uncertainties

The goals for this section of your report are

  1. Show us what happens to a raw data point prior to being reported, such that the raw data can be analyzed by someone, somewhere else.
  2. Show us where uncertainties of the reported values come from (i.e. bias error, precision error, etc.)
  3. Quantify how confident you are in the reported measurements.
  4. Conform to ASTM Standard E2586 – Standard Practice for Calculating and Using Basic Statistics. You can download the pdf for free while on campus.

How do we quantify confidence? This is where we realize the value of our engineering education. We’ve all calculated a standard deviation by now and know that 68% of the measurements fall within ±σ of the mean (a.k.a. a coverage factor of 1), 95% lie within ±2σ (a.k.a. a coverage factor of 2), and 99.7% lie within ±3σ (a.k.a. a coverage factor of 3). While it’s straight forward to calculate the coverage factor for every steady state measurement you take. You should also do a repeatability test at several points to double check. A classic example is that someone states that there uncertainty has a coverage factor of 2 (99.7% of all measurements fall with y of mean), however this is only a repeatability test of the precision error. You still need to propogate instrument uncertainties to determine the bias error.

Precision versus Bias error

 

This portion of the lesson contains a considerable amount of math as we go through the Root-Sum-Square (RSS) method and what it means for uncertainty propogation. Remember that we can do this math by hand or with EES, which will estimate the uncertainties numerically. Here’s a link to the hand-written notes: lesson-6-lecture-notes.

ME 406 Lesson 5: Instrument Calibration and Traceability

In section 4.1 we created a table of key instruments for our measurements that included columns for instrument, purpose, make, model number, range, and uncertainty. Today we dive into the details of instrument uncertainty and traceability.

Many times knowing the precision/uncertainty of your measurement is just as, if not more, important than the value of the measurement itself. The question is: how can you quantify your confidence in the measurement? Two kinds of error will affect your instruments: Precision error and bias error.

Precision versus Bias error

For the most part, we can resolve precision error with statistics. We’ll cover that next time. So how do we minimize bias error to the greatest extent possible?

Calibration: determining and documenting the deviation of a measuring instruments indication from the conventional ‘true’ value.

Sure we can calibrate our instruments so that we know they are accurate. But how do we know the calibrations are correct? What value is ‘true’ and how can we trust it?

Traceability: process whereby the indication of a measuring instrument (or a material measure) can be compared, in one or more stages, with a national standard for the measurement in question.

In the United States national standards are maintained and improved by the National Institute of Standards and Technology (NIST). NIST may easily be the most under-appreciated federal agency as a NIST standard is used for nearly every custody exchange in our economy. Formerly known as the National Bureau of Standards (NBS), the organization was mandated by congress after the civil war to standardize the exchange of sugar as a commodity. As it turns out, it’s not easy to standardize the purity of white powdery substance for custody exchange. A full history of NIST and the NBS can be found here. I’ve worked with NIST researchers for my entire career, and spent several summers at NIST-Boulder. It’s a wonderful place!

So if NIST keeps the standards, how do our instruments get compared to this standard? Here’s a diagram of traceability levels established by the European Cooperation for Accreditation of Laboratories (EAL-G12):

Traceability pyramid

This traceability pyramid above is a handy way of understanding the problem of traceability. The wider the base, the more instruments in the world that exist in that level of the pyramid and the cheaper, or lower quality the measurement calibration is. For example THE kilogram block that all other kilograms are based on, is kept in a safe under very carefully controlled conditions, I call this the very top of the pyramid, or level 0. Level 0 is used to create the primary standards sent around the world for each country’s National Standards, level 1, that are implemented by the national standard body. Level 2 is a reference standard that is created from the primary standard and distributed to accredited calibration laboratories around the country. These in turn create workplace/factory standards, Level 3, that calibrate the instruments used by the workplace/factory for actually producing things at Level 4. Here’s an example of a length measurement standard applied to the traceability pyramid, again from EAL-G12.

Traceability pyramid for length

As you can see, an ultimate micrometer or dial gauge isn’t necessarily kept in a safe in Switzerland to calibrate all other dial gauges like in the case of the kilogram. The actual type of instrument can change depending on standard level (0,1,2,3,4). This makes sense as the level of precision goes up significantly with standard level, in many cases you just need different physics paradigms to achieve this.

As you can see, the process of traceability can quickly become complex! As the table above shows, most standards bodies will provide a certificate of traceability to describe this chain of calibration on a single convenient document. Here’s an example of such a certificate of traceability for the microphones on the loudspeaker project:

Microphone Traceability Certificate

There is an incredible amount of information on this document! Key points include the actual instruments, including serial numbers used in the unbroken chain going back to the primary standard. The dates of calibrations, including environmental conditions are provided, along with the calibration curve itself. In the fine print you’ll also see the obligatory, “whose accuracies are traceable to the National Institute of Standards and Technology.” As you can guess, this document is valueable, sometimes more valuable than the instrument itself. Hang onto these in a safe place!

But even with this document things are not fine and dandy. Is it current enough? How do you know the instrument was not abused before you started use? These are complex questions that there isn’t currently a standard for.

A few years ago I proposed a Standards Traceability Index (STI) to go along with the education of traceability. The STI has three numbers: X.XX

X. — This is the standard level (from the pyramid above) for the instrument that you are using (almost everything in our lab is level 4, although the calibration instruments are level 3).

.X — This number describes the status of the instrument’s traceability. For example, 0 is used for a current unbroken chain, 1 for a current but broken chain, 2 for an outdated unbroken chain, 3 for an outdated broken chain, 4 for no certificate chain.

._X — This number describes the status/condition of the instrument. For example, 0 is brand new and performance is validated, 1 is used by performance is validated, 2 is new but non-validated, 3 is used and non-validated, 4 is for unknown condition.

So for example, a brand new platinum resistance thermometer with a NIST traceable calibration that you just dunked in  a liquid nitrogen bath to check would likely have an STI = 4.00.

Using the STI for this class is likely overkill. But if you need to do a very good job, it’s good to think about. In general you should include a statement on the traceability for all your instruments when assessing your measurement confidence/uncertainty. Better yet, check the calibrations once you’re finished with one of our laboratory calibration standards. If you’re uncertain in your results, than how valuable are they?

Traceability is also important for use of property correlations and flow correlations. It’s sadly very common for people to reference a software package as the source of their property data. In reality though the software is just implementing an equation that someone spent a lot of time on. Reference the original equations, not the software. Many people end up embarrassed when they cite some fancy software package, only to find out it’s using the ideal-gas law in a non-ideal situation.

ME 406 Lesson 4: Experimental Setup and Procedure

Now that we have our motivation for an experiment established (Chapter 1), showed that there is a gap needing to be filled in the literature (Chapter 2), and have a working modeling connecting what our client cares about to what we are measuring (Chapter 3), it’s time to start experimenting. Give us an introductory paragraph describing how this experimental chapter is organized. Begin with the following section:

4.1 Experimental Setup

The goals of the experimental section are two:

  1. SHOW that you understand the key components of the experiment and how they work.
  2. SHOW enough information so that the experiment can be repeated by someone else, somewhere else without having to contact you.

Think about the significance of that for a minute. Have you ever heard about an experimental study where the results could not be repeated/validated? What happens to the credibility of the engineers/researchers who published the report? Have you ever heard about an accident or a near miss where a carefully documented experiment and report saved the engineer’s job? (e.g. They had it right in the report, but a technician cut a corner.) This part of the report, just like careful citation of appropriate standards, can be very important to saving you in a lawsuit or litigation. In Chapter 2 you simply state that the standards cover certain areas. This is the part of Chapter 4 where you SHOW that you are following/conforming to the standards.

Here’s a few good ways to show us you understand the components of an experiment:

  1. A table of instruments involved in recording the data. Give us the purpose, make, model #, applicable ranges, and uncertainties.
  2. Diagrams and accompanying pictures of the actual experiment. Potential diagrams include flow, wiring, force, energy, and others. These are commonly called Plumbing and Instrumentation (P&ID) diagrams and are often referred to in incident investigations.

Making good diagrams is a design challenge in itself. One common mistake is to take a picture of your experiment and superimpose numbers/labels over the picture to identify components. This is usually none optimal. There is an old saying, “A picture is worth a thousand words.” I counter that with “Nine hundred of which are irrelevant.” The Google Maps approach to have a layer with just the streets, then another layer to add in the satellite view, is likely the best. For your reports, I recommend having a diagram/schematic view that is accompanied by an actual picture in a similar orientation as your schematic so your reader can quickly go from one to the other.

Our library has a copy of the ANSI/ISA 2009 Instrumentation Symbols and Identification standard. A briefer version is available on the P&ID Wikipedia page. Whatever organization you end up working for is likely to have their own in-house standard protocol for P&ID that you’ll have to follow. There are several software programs, including Inventor, that will automatically make a P&ID for you. In general a P&ID will have the following features: A) Key piping connections and instrument locations/details, and B) Critical safety, control, and shutdown schemes. You should also have a narrative that walks us through the visuals to make sure that we don’t miss the key features/points/takeaways. How you accomplish this is up to you. Some students have done incredible work using the WSU free access to Microsoft Visio. Here are a few of the very best examples I’ve seen from over the years:

Roots Blower Schematic

Centrifugal Fan Schematic

So as you can see, this could take some time to do very well. But once you’ve got a visual like this, it serves as a key feature in your reports, actual experimental info, and presentation. I’ve witnessed visuals like this drop jaws — that’s a lot of political capital in your favor if you can do it to your boss.

4.2 Experimental Procedure

This is a start to finish, step-by-step, enumerated description of how the experiment is started, a measurement taken, and shutdown. Often these test procedures will be given to untrained technicians for data collection so it’s important that you write it such that nobody can get hurt. You’ll want to use your labeled/numbered diagram as a reference throughout. Brief, informative, friendly, and firm… Relevant, credible, efficient…

Next time we’ll cover how we actually analyze and interpret the raw data to make it reportable, with quantified confidence/uncertainty. But for now, you’re ready to write your proposed test plan. Here’s a prompt and grading rubric.

406 A Test Plan Prompt

406 A Test Plan Rubric

 

ME 406 Lesson 3: Using Theory to Guide Experiments

In just about every job I’ve been in, people were tempted to label me as either a “theorist” or an “experimentalist”. — Don’t take the bait. It’s easy to fit into a stereotype, tough to break free of them. The very best engineers are competent with both the theory and the experiment. It’s what we call a positive synergy — knowledge of one aids the other.

This brings us to three general guiding principles for the Theory chapter of our reports:

  1. Relevance: Connect the primary motivations/needs/objectives for the experiment (performance, efficiency, flow rate, etc.) to the key variables for the experiment.
  2. Efficiency: Establish only enough of the theory such that the calculations can be repeated by a fellow engineer without the need to contact you.
  3. Credibility: At the end of your project, your theoretical predictions should agree with your experimental measurements and you should have justifications for limits to and deviations from theory.

Related to the first principle, when constructing experiments from scratch it is important to determine your key variables a priori (in advance) of building the experiment. This allows you to focus your time and money on the key variables that matter to the client/customer goal.

Example: My Ph.D. dissertation was on the modeling of visco-plastic flow of solid hydrogenic fuel within twin-screw extruders for the fueling of fusion energy tokomaks. Nobody had built a machine to solidify and extrude solid hydrogen before. Only the most very basic material properties of hydrogen were available! How do we most efficiently build an experiment that helps us develop theory to model how such a machine will operate? Here’s a visual of my first research poster on this subject:

By building a simple numerical heat exchanger model of the vortex tube and varying the key input parameters/variables, we discovered that the two-phase heat transfer, latent heat, and viscous dissipation. The latent heat was known and the two-phase heat transfer coefficient was not necessarily controllable, so the most important parameter for us to know very well in the model was the viscous dissipation. So we built an experiment to measure that very well, and along the way measured the 2-Phase heat transfer coefficient among other things.

We had a motivated client with a need, searched the literature to know that it was a gap in knowledge, and completed a careful calculation to show that an experiment will help solve the need. Now, before we spent considerable funds on equipment and instruments, we had a very good reason to do the experiment and knew what we needed to measure. So, how do we go about actually doing our theory/analysis?

Here’s a few modeling steps:

  1. If you are still using a calculator, stop. Calculators are typewriters. Download Engineering Equation Solver (EES) at mme.wsu.edu/ees It’s one of many equation processors (similar to a word processor).
  2. Take whatever units you have and immediately convert everything to base SI (m,s,kg,Pa,etc.). Base SI is the only self-consistent unit system. Read my post – End US Engineering Education of English units for more. It’s a huge loss to our national economy that gets worse every year. You will solve your problems faster, with fewer mistakes in base SI. EES will easily convert and check your units are consistent for you. Convert your final plots into whatever units your clients can best handle.
  3. Conduct an Uncertainty/Sensitivity analysis of your programmed equations in EES (show heat transfer example).
  4. Determine key performance metrics for your experiment. Remember how most performance metrics are determined:

System Performance = (What you want)/(What you paid to get it)

Device/Component Performance = (What you got)/(What you could’ve)   a.k.a. (actual)/(ideal)

With these programmed, you can now make performance/design curves on plots that can help you to determine where to take measurements. Once validated, these curves save substantial money and time.

Being able to quantitatively show where the losses that caused the actual to be worse than the ideal tells you know how best to improve the system and gives you values as an engineer. Once completed, the report will naturally transition to the need to actually do the experiment to test the theory.

“No matter how bright you are or clever your theory is, if it doesn’t agree with experiment, it’s wrong.” ~Richard Feynman

“Through measuring is knowing.” ~Heiki Kammerlingh-Onnes

 

ME 406 Lesson 1- The Introduction

Welcome to Northwest Engineering Inc. I’m the CEO, boss, and your supervisor. Here at NEI we characterize the performance of large mechanical machinery for use in other engineering systems.

Before we tour the lab and assign projects, you need to know how the class is structured. Here’s a link to the course syllabus with the schedule: ME 406 syllabus F16.

During this first Engineering Experimentation third of the class, each class lecture corresponds to the portion of your report and presentation you need to complete. Let that be clear, if you keep up with the class lectures, the work of completing your report and presentation will be spread out over the next month. The amount of work you need to do for these reports and presentations is extensive — in many ways like running a marathon. It’s up to you whether you want to run the marathon in increments over the next month, or attempt to do it the night before… and I don’t award misery points.

Welcome to Chapter 1 — The Introduction

99.5% of readers finish the first sentence of a report however is often overlooked by undergraduate engineers (Holtz et al. 1994). Utilizing citations to hard statistics justifying your point builds relevance and credibility. Keeping this first sentence, and those thereafter, less than 22 words is a good start.

In this first Chapter you need to empathize your readers to your cause. You need to show your primary audience/reader that you understand the problem and their needs. Don’t simply TELL US you understand, SHOW US you understand by doing a thorough job of reading and quantifying the significance of the problem. Pandering gets annoying very quickly. For example, “We’ve read the project requirements and fully understand what’s at stake.” — That’s the kind of desperate writing I see all of the time when people waited too long to write their reports and use filler the night before. Don’t waste your time!

Go to Google Scholar and find a few articles or theses documents to see what opening sentences and paragraphs appeal to you and which ones do not. As you’ll see, there’s a natural progression here. A good Introduction Chapter sets up the need to review the prior work, standards, and competing design paradigms in a thorough literature review. Hence Chapter 2 — Background Literature. From there, a clear need to investigate a specific concept, and necessary equations will lead to Chapter 3 — Theory. The theory section is useful to show you can validate your theoretical predictions with an experiment, hence Chapter 4 — Experiment. Ultimately you’ll conclude with a thorough comparison of the theory and experimental results, that will hopefully solve the needs and concerns of your client/boss, Chapter 5 — Results and Conclusions.

Now let’s partner up for our A projects. In the past, we’ve defined partners for A projects based on GPA, however I’ve found through experience that nearly all metric-based attempts at team formation fail. The best way to form a team is to allow you to use your judgement. You know who you work best with, and this will be a lot of work.

Here’s the list of projects we’re going to run as “A” — ME 406 Project A assignments. I want you to vote for each project with a 1, 2, or 3. Discuss these with your partner and we’ll have you mark them on the board.

Now let’s go down to the lab space and do a safety review before you get started. Here’s the safety form I developed with WSU Environmental Health and Safety — ME 406 Safety Checklist and FMA. These are, indeed, powerful machines and they can break. The reality is we don’t have the resources in our department to continuously service them — this is a reality in many work places. As an engineer, you need to make it a practice to conduct a safety overview and Failure Modes and Effects Analysis (FMEA) before you operate a machine. Not just for your own safety, but the safety of the machine as well.

Washington State University