Engineering Scrutiny: Scrutinizing Analytical Formulas, Results and Computer Solutions

“Be your own worst critic, unless you prefer that someone else be your worst critic.”

- I dunno, I just made it up. But, it doesn’t sound very original.

Scrutinizing Analytical Formulas and Results

I often see analyses that I can tell within 5 seconds must be wrong. I have three tests, which should be done in the order listed, for checking and verifying results. These tests will weed out 95% of all mistakes. I call these the “smoke test,” “function test,” and “performance test,” by analogy with building electronic devices.

1. Smoke Test. In electronics, this corresponds to turning the power switch on and seeing if the device smokes or not. If it smokes, you know the device can’t possibly be working right (unless you intended for it to smoke.) In analytical engineering terms, this corresponds to checking the units. You have no idea how many results people report that can’t be correct because the units are wrong (i.e. the result was 6 kilograms, but they were trying to calculate the speed of something.) You will catch 90% of your mistakes if you just check the units. For example, if I just derived the ideal gas law for the first time and predicted PV = nℜ/T you can quickly see that the units on the right-hand side of the equation are different from those on the left-hand side. There are several additional rules that must be followed:

• Anything inside a square root, cube root, etc. must have units that are a perfect square (e.g. m² /sec²), cube, etc.) This does not mean that every term inside the square root must be a perfect square, only that the combination of all terms must be a perfect square. For example, the speed (v) of a frictionless freely falling object in a gravitational field is v=√2gh , where g = acceleration of gravity (units length/time² ) and h is the height from which the object was dropped (units length). Neither g nor h have units that are a perfect square, but when multiplied together the units are (length/time²)(length) = length² /time² , which is a perfect square, and when you take the square root, the units are v =√length² time² = length time as required.

• Anything inside a log, exponent, trigonometric function, etc., must be dimensionless (I don’t know how to take the log of 6 kilograms). Again, the individual terms inside the function need not all be dimensionless, but the combination must be dimensionless.

• Any two quantities that are added together must have the same units (I can’t add 6 kilograms and 19 meters/second. Also, I can add 6 miles per hour and 19 meters per second, but I have to convert 6 miles per hour into meters per second, or convert 19 meters per second into miles per hour, before adding the terms together.)

2. Function Test. In electronics, this corresponds to checking to see if the device does what I designed it to do, e.g. that the red light blinks when I flip switch on, the meter reading increases when I turn the knob to the right, the bell rings when I push the button, etc. – assuming that was what I intended that it do. In analytical terms this corresponds to determining if the result gives sensible predictions. Again, there are several rules that must be followed:

• Determine if the sign (+ or -) of the result is reasonable. For example, if your prediction of the absolute temperature of something is –72 Kelvin, you should check your analysis again.

• For terms in an equation with property values in the denominator, can that value be zero? (In which case the term would go to infinity). Even if the property can’t go to zero, does it make sense that as the value decreases, the term would increase?

• Determine whether what happens to y as x goes up or down is reasonable or not. For example, in the ideal gas law, PV = nℜT:

o At fixed volume (V) and number of moles of gas (n), as T increases then P increases – reasonable

o At fixed temperature (T) and n, as V increases then P decreases – reasonable

o Etc.

• Determine what happens in the limit where x goes to special values, e.g. zero, one or infinity as appropriate. For example, consider the equation for the temperature as a function of time T(t) of an object starting at temperature T i at time t = 0 having surface area A (units m²), volume V (units m³), density ρ (units kg/m³) and heat capacity C_P (units J/kg˚C) that is suddenly dunked into a fluid at temperature T_∞ with heat transfer coefficient h (units Watts/m 2 ˚C). It can be shown that in this case T(t) is given by

hA/ρVC P has units of (Watts/m²˚C)(m²)/(kg/m³)(m³)(J/kg˚C) = 1/s, so (hA/ρVC_P)t is dimensionless, thus the formula easily passes the smoke test. But does it make sense? At t = 0, T_i = 0 as expected. What happens if you wait for a long time? The temperature can reach T_∞ but cannot overshoot it (a consequence of the Second Law of Thermodynamics). In the limit t → ∞, the term exp(-(hA/ρVC_P)t) goes to zero, thus T → T_∞ as expected. Other scrutiny checks: if h or A increases, heat can be transferred to the object more quickly, thus the time to approach T_∞ decreases. Also, if ρ, V or C_P increases, the “thermal inertia” (resistance to change in temperature) increases, so the time required to approach T_∞ increases. So, the formula makes sense.

• If your formula contains a difference of terms, determine what happens if those 2 terms are equal. For example, in the above formula, if T_i = T_∞ , then the formula becomes simply T(t) = T ∞ for all time. This makes sense because if the bar temperature and fluid temperature are the same, then there is no heat transfer to or from the bar and thus its temperature never changes (again, a consequence of the Second Law of Thermodynamics … two objects at the same temperature cannot exchange energy via heat transfer.)

3. Performance Test. In electronics, this corresponds to determining how fast, how accurate, etc. the device is. In analytical terms this corresponds to determining how accurate the result is. This means of course you have to compare it to something else that you trust, i.e. an experiment, a more sophisticated analysis, someone else’s published result (of course there is no guarantee that their result is correct just because it got published, but you need to check it anyway.) For example, if I derived the ideal gas law and predicted PV = 7nRT, it passes the smoke and function tests with no problem, but it fails the performance test miserably (by a factor of 7). But of course the problem is deciding which result to trust as being at least as accurate as your own result; this of course is something that cannot be determined in a rigorous way, it requires a judgment call based on your experience.

Scrutinizing Computer Solutions

(This part is beyond what I expect you to know for AME 101 but I include it for completeness)

Similar to analyses, I often see computational results that I can tell within 5 seconds must be wrong. It is notoriously easy to be lulled into a sense of confidence in computed results, because the computer always gives you some result, and that result always looks good when plotted in a 3D shaded color orthographic projection. The corresponding “smoke test,” “function test,” and “performance test,” are as follows:

1. Smoke Test. Start the computer program running, and see if it crashes or not. If it doesn’t crash, you’ve passed the smoke test, part (a). Part (b) of the smoke test is to determine if the computed result passes the global conservation test. The goal of any program is to satisfy mass, momentum, energy and atom conservation at every point in the computational domain subject to certain constituitive relations (e.g., Newton’s law of viscosity τ x = µ∂u x /∂y), Hooke’s Law σ = Eε) and equations of state (e.g., the ideal gas law.) This is a hard problem, and it is even hard to verify that the solution is correct once it is obtained. But it is easy to determine whether or not global conservation is satisfied, that is,

• Is mass conserved, that is, does the sum of all the mass fluxes at the inlets, minus the mass fluxes at the outlets, equal to the rate of change of mass of the system (=0 for steady problems)?

• Is momentum conserved in each coordinate direction?

• Is energy conserved?

• Is each type of atom conserved?

If not, you are 100% certain that your calculation is wrong. You would be amazed at how many results are never “sanity checked” in this way, and in fact fail the sanity check when, after months or years of effort and somehow the results never look right, someone finally gets around to checking these things, the calculations fail the test and you realize all that time and effort was wasted.

2. Performance Test. Comes before the function test in this case. For computational studies, a critical performance test is to compare your result to a known analytical result under simplified conditions. For example, if you’re computing flow in a pipe at high Reynolds numbers (where the flow is turbulent), with chemical reaction, temperature-dependent transport properties, variable density, etc., first check your result against the textbook solution that assumes constant density, constant transport properties, etc., by making all of the simplifying assumptions (in your model) that the analytical solution employs. If you don’t do this, you really have no way of knowing if your model is valid or not. You can also use previous computations by yourself or others for testing, but of course there is no absolute guarantee that those computations were correct.

3. Function Test. Similar to function test for analyses. By the way, even if you’re just doing a quick calculation, I recommend not using a calculator. Enter the data into an Excel spreadsheet so that you can add/change/scrutinize/save calculations as needed. Sometimes I see an obviously invalid result and when I ask, “How did you get that result? What numbers did you use?” the answer is “I put the numbers into the calculator and this was the result I got.” But how do you know you entered the numbers and formulas correctly? What if you need to re-do the calculation for a slightly different set of numbers?

Examples of the Use of Units and Scrutiny

These examples, particularly the first one, also introduce the concept of “back of the envelope” estimates, a powerful engineering tool.

Example 1. Drag force and power requirements for an automobile

A car with good aerodynamics has a drag coefficient (C_D) of 0.3. The drag coefficient is defined as the ratio of the drag force (F_D) to the dynamic pressure of the flow = ½ρv² (where ρ is the fluid density and v the fluid velocity far from the object) multiplied by the cross-section area (A) of the object, i.e.

The density of air at standard conditions is 1.18 kg/m³.

(a) Estimate the power (in units of horsepower) required to overcome the aerodynamic drag of such a car at 60 miles per hour.

(b) Estimate the gas mileage of such a car. The heating value of gasoline is 4.3 x 10⁷ J/kg and its density is 750 kg/m³.

Why is this value of miles/gallon so high?

o The main problem is that conversion of fuel energy to engine output shaft work is about 20% efficient at highway cruise conditions, thus the gas mileage would be 199.0706726 x 0.2 = 39.81413452 mpg

o Also, besides air drag, there are other losses in the transmission, driveline, tires – at best the drivetrain is 80% efficient – so now we’re down to 31.85130761 mpg

o Also – other loads on engine – air conditioning, generator, …

What else is wrong? There are too many significant figures; at most 2 or 3 are acceptable. When we state 31.85130761 mpg, that means we think that the miles per gallon is closer to 31.85130761 mpg than 31.85130760 mpg or 31.85130762 mpg. Of course we can’t measure the miles per gallon to anywhere near this level of accuracy. 31 is probably ok, 31.9 is questionable and 31.85 is ridiculous. You will want to carry a few extra digits of precision through the calculations to avoid round-off errors, but then at the end, round off your calculation to a reasonable number of significant figures based on the uncertainty of the most uncertain parameter. That is, if I know the drag coefficient only to the first digit, i.e. I know that it’s closer to 0.3 than 0.2 or 0.4, but not more precisely than that, there is no point in reporting the result to 3 significant figures.

Example 2. Scrutiny of a New Formula

I calculated for the first time ever the rate of heat transfer (q) (in Watts) as a function of time t from an aluminum bar of radius r, length L, thermal conductivity k (units Watts/m˚C), thermal diffusivity α (units m² /s), heat transfer coefficient h (units Watts/m²˚C) and initial temperature T_bar conducting and radiating to surroundings at temperature T_∞ as

Using “engineering scrutiny,” what “obvious” mistakes can you find with this formula? What is the likely “correct” formula?

1. The units are wrong in the first term (Watts/m, not Watts)

2. The units are wrong in the second term inside the parenthesis (can’t add 1 and something with units of temperature)

3. The first term on the right side of the equation goes to infinity as the time (t) goes to infinity – probably there should be a negative sign in the exponent so that the whole term goes to zero as time goes to infinity.

4. The length of the bar (L) doesn’t appear anywhere

5. The signs on (T_bar – T_∞) are different in the two terms – but heat must ALWAYS be transferred from hot to cold, never the reverse, so the two terms cannot have different signs. One can, with equal validity, define heat transfer as being positive either to or from the bar, but with either definition, you can’t have heat transfer being positive in one term and negative in the other term.

6. Only the first term on the right side of the equation is multiplied by the e^{(− α t /r2 )} factor, and thus will go to zero as t → ∞. So the other term would still be non-zero even when t → ∞, which doesn’t make sense since the amount of heat transfer (q) has to go to zero as t → ∞. So probably both terms should be multiplied by the e^{(− α t /r2 )} factor.

Based on these considerations, a possibly correct formula, which would pass all of the smoke and function tests is

q = [kL(T_bar −T_∞ )+hr² (T_bar −T_∞ )]e^{− α t/r2}

Actually even this is a bit odd since the first term (conduction heat transfer) is proportional to the length L but the second term (convection heat transfer) is independent of L … a still more likely formula would have both terms proportional to L, e.g.

q = [kL(T_bar −T_∞ )+hrL (T_bar −T_∞ )]e^{− α t/r2}

Example 3. Thermoelectric Generator

The thermal efficiency (η) = (electrical power out) / (thermal power in) of a thermoelectric power generation device (used in outer planetary spacecraft (Figure 2), powered by heat generated from radioisotope decay, typically plutonium-238) is given by

where T is the temperature, the subscripts L, H and a refer to cold-side (low temperature), hot-side (high temperature) and average respectively, and Z is the “thermoelectric figure of merit”:

where S is the Seebeck coefficient of material (units Volts/K, indicates how many volts are produced for each degree of temperature change across the material), ρ is the electrical resistivity (units ohm m) (not to be confused with density!) and k is the material’s thermal conductivity (W/mK).

(a) show that the units are valid (passes smoke test)

Everything is obviously dimensionless except for ZTₐ , which must itself be dimensionless so that I can add it to 1. Note

(b) show that the equation makes physical sense (passes function test)

o If the material Z = 0, it produces no electrical power thus the efficiency should be zero. If Z = 0 then

o If T_L = T_H, then there is no temperature difference across the thermoelectric material, and thus no power can be generated. In this case

o Even the best possible material (ZTₐ → ∞) cannot produce an efficiency greater than the theoretically best possible efficiency (called the Carnot cycle efficiency, see page 86) = 1 – T_L / T_H, for the same temperature range. As ZTₐ → ∞,

Side note #1: a good thermoelectric material such as Bi₂ Te₃ has ZTₐ ≈ 1 and works up to about 200˚C before it starts to melt, thus

By comparison, your car engine has an efficiency of about 25%. So practical thermoelectric materials are, in general, not very good sources of electrical power, but are extremely useful in some niche applications, particularly when either (1) it is essential to have a device with no moving parts or (2) a “free” source of thermal energy at relatively low temperature is available, e.g. the exhaust of an internal combustion engine.

Side note #2: a good thermoelectric material has a high S, so produces a large voltage for a small temperature change, a low ρ so that the resistance of the material to the flow of electric current is low, and a low k so that the temperature across the material ΔT is high. The heat transfer rate (in Watts) q = kAΔT/Δx where A is the cross-sectional area of the material and Δx is its thickness. So for a given ΔT, a smaller k means less q is transferred across the material. One might think that less q is worse, but no. Consider this:

The electrical power = IV = (V/R)V = V² /R = (SΔT)² /(ρΔx/A) = S² ΔT² A/ρΔx.

The thermal power = kAΔT/Δx

The ratio of electrical to thermal power is [S²ΔT²A/ρΔx]/[kAΔT/Δx] = (S² /ρk)ΔT = ZΔT, which is why Z is the “figure of merit” for thermoelectric generators.)

Radioisotope thermoelectric generator used for deep space missions. Note that the plutonium-238 radioisotope is called simply, “General Purpose Heat Source.”

Example 4. Density of Matter

Estimate the density of a neutron. Does the result make sense? The density of a white dwarf star is about 2 x 10¹² kg/m³ – is this reasonable?

The mass of a neutron is about one atomic mass unit (AMU), where a carbon-12 atom has a mass of 12 AMU and a mole of carbon-12 atoms has a mass of 12 grams. Thus one neutron has a mass of

A neutron has a radius (r) of about 0.8 femtometer = 0.8 x 10⁻¹⁵ meter. Treating the neutron as a sphere, the volume is 4πr³ /3, and the density (ρ) is the mass divided by the volume, thus

By comparison, water has a density of 10³ kg/m³ , so the density of a neutron is far higher (by a factor of 10¹⁴) than that of atoms including their electrons. This is expected since the nucleus of an atom occupies only a small portion of the total space occupied by an atom – most of the atom is empty space where the electrons reside. Also, even the density of the white dwarf star is far less than the neutrons (by a factor of 10⁵), which shows that the electron structure is squashed by the mass of the star, but not nearly down to the neutron scale (protons have a mass and size similar to neutrons, so the same point applies to protons too.)