Myth vs Science
IMHO Like already stated, would possibly be more beneficial to go about things in a more scientific manner. Think about the questions posed as well then conduct a real test.
Let me share this excellent article.
----------------------------------
Testing and Trialing
by Dr. Lynette Morgan
2010-08-01
Not all of us are science geeks, but sometimes we need science to help us make important decisions about how we grow our crops. Horticulture is one branch of applied science that there is still much to learn about and even small growers can discover and investigate new ideas. Most of us have tried a new product, nutrient formulation, growth promotant or pest spray to see how well it performs on our plants, but often the results are just speculation. Natural variation within biological systems involving plants is so common that determining if there is an actual effect can be difficult unless the set-up is precise and correct in the first place. However, with good planning, knowledge of the correct scientific method and some basic math, accurate testing and trialing is possible.
The dangers of informal comparisons
Unfortunately many bad decisions and misinterpretations have been made because of informal plant tests. Applying a new product to be tested and ‘seeing how it goes’ doesn’t generate conclusive answers and may give an inaccurate result as to the effectiveness of the treatment. Tests must be compared to something else for the results to be relevant. The new idea, product or system is termed the ‘treatment’ and what it is compared against is called the ‘control.’ The control is a separate set of plants grown under the same conditions, to which the treatment is not applied. The treatment plants and control plants form the basis of the trial and accurate comparisons can be made.
Natural variations in light, temperature and humidity tend to occur accross most growing environments, so experimental design is important to help prevent bias.
Step 1: Ask a question
All good experiments originate from a basic question. For instance, ‘Will this new product make my crop grow faster or producer higher yields?’ Depending on your production philosophy, other questions might involve wanting to know if organic pest control methods are effective, or if water treatment reduces the occurrence of pythium. Asking such questions leads to forming the hypothesis of the experiment—a statement about what the predicted result of the experiment might be. The experiment either proves or disproves the hypothesis.
Step 2: Restrict biological bias
The problem with plant trials is that plants are naturally quite variable. Even within a crop sown at the same time, of the same cultivar and grown in the same environment, natural variations will exist in growth and yields from plant to plant. With a correctly run trial, we want to reduce this natural variation as much as possible; otherwise we might incorrectly assume that some naturally occurring differences were caused by the treatment we applied and assume the wrong conclusions. A good example of natural bias is taking one or two plants and applying different treatments to each. Many plants are required to ensure that the differences are more likely to be due to the treatment rather than just some natural variation between a small set of individual plants. Another common mistake is putting all of the treatment plants in a group and putting the control plants in a separate group across the other side of the growing area. There are always slight differences in temperature, humidity and light within a growing area, so any differences between the two sets of plants might be due to slight differences in their growing environment, rather than an indication of the treatment results.
Step 3: Replication and randomization
To try and reduce the natural bias that plant experiments are prone to, replication and randomization are used. In a randomized experimental design, plants or plots of plants are randomly assigned to an experimental group of treatment. Randomization is the most reliable way of creating treatment groups that all start out exactly the same. It prevents the largest seedlings or heaviest plants from being selected for one treatment, while smaller plants end up in another—a major source of bias that gives incorrect results.
Replication is equally important; sufficient replication improves the significance of the result and reduces result variability. Generally in small experiments, three or more replications are used, with each replication having at least six to 10 plants. For example, in a basic experiment involving tomatoes there would be two treatments: (i) the old nutrient product, which is the control treatment and (ii) the new nutrient product to be tested. Each of these treatments (control and new nutrient) would have three sets (replications) each containing six or more plants. This gives a total of six sets of six plants. Once the data from this is obtained it can be easily analyzed to determine if the new nutrient had a significant effect or not. This is far more accurate than simply dividing six plants into two separate treatment groups as the replications help eliminate some of that natural variation plants are prone to.
Each of the three reps of six plants would then be randomly assigned a position in the growing area. When the plants are measured or assessed, the data from each treatment replication is kept separate so that some statistics can be carried out to determine if differences caused by the new treatment actually exist.
Once the trial is up and running it is vital that, apart from the treatment being applied, all plants in the experiment are treated exactly the same. This means they are given the same amount of light, water, nutrients, pest and disease control (if needed during the trial) and any other growth factors, so as not to compromise the trial. The different treatments and replications need to be grown at the same time and they must be of the same species and cultivar (unless different species and cultivars are the actual trial).
“Even when we try to be neutral, we tend to be influenced by what we want to see.”
Some growers choose to run their experiment more than once. This is a good idea if using a greenhouse or outdoor crop as seasons will affect the results of many trials. Running more than one trial over time can also confirm results of a certain treatment, which is worth doing to add weight to a product claim or a new idea being tested.
Unfortunately many good plant trials have been wasted by not carrying out the correct assessment or measurements. Simply ‘eye-balling’ plants to see if there are any visible differences between treatments often gives rather inaccurate results, particularly if the treatments applied have highly visible labels so the assessor knows which is the control and which plants have been given the new treatment. Even when we try to be neutral, we tend to be influenced by what we want to see. Hoping a new treatment will grow faster can lead to the assessor seeing slight differences in height that are not actually there. Also, many successful treatments often have effects that can’t be seen by looking at the plants. Plants of the same height may have one treatment that is heavier or has a greater dry weight, or better tasting fruit, all of which can’t be measured by just looking at the treatments. Determining what to measure and using analytical data (weights, lengths, leaf area index, chemical composition, nutritional analysis, etc.) gives more accurate results than just having a look-see at the plants.
Step 4: Recording
Every aspect of an experiment should be recorded: observations, applications, measurements, calculations, conclusions, etc. This serves two purposes—it allows others to follow your method and achieve the same results and it allows the trial to be reviewed in case of an unexpected result. Sometimes unintended biases can occur in a trial and often reviewing records can help the grower work out what went wrong. Taking photographs of any treatment differences or unusual occurrences is also useful for future reference.
Part of the experimental design is careful consideration of what variables to measure. There are obvious measurements such as plant weight at harvest or fruit yields per week, but sometimes determining the most meaningful data can be difficult. Plant height is not necessarily an indication of growth rate or productivity. Sometimes shorter, more branched plants produce more fruit than taller, leaner ones and plant fresh weight may not be that meaningful if testing a product that claims to increase fruit flavor. Fruit or vegetable flavor assessment can be a minefield as taste tests need to be run correctly and by many different trained panelists to get a true indication of flavor improvement. There are also analytical tests such as brix for sweetness, which can be easily carried out by growers running small trials. Often in horticultural trials, percentage of dry plant matter is used as a better indication of increased photosynthesis and biomass production. With hydroponic experiments it is always a good idea to keep track of EC, pH and water usage between different treatments. When trialing pest and disease control products other factors may be more relevant such as the number of live or dead insects after spray application, size of disease lesions, spray damage occurrence as well as overall yields and plant performance.
Step 5: What to do with the data
Once the data has been collected from a trial—plant weight, height, leaf area, yield or any other ‘hard’ data—the first step to analyzing the results is to find the mean or average from each treatment set. Just eye-balling a set of measurements and trying to decide which is highest, or totaling them, is not going to give an accurate answer. The mean value per treatment is required. The mean is calculated by adding up all the numbers for each treatment and dividing it by the number of plants in that treatment. Technically, with a true scientific trial, we would not just stop at working out the mean of each treatment. While it gives us a rough indication of differences that might be significant, it is the variance of the data around the mean that gives the final answer as to whether a treatment had a relevant effect or not. It is not difficult to work out the variance and standard error of a set of data and is usually well covered in basic math courses. For those of us for whom school was quite a long time ago and who need a reminder the following links and references detail the process.
A well thought out and run trial with correctly analyzed data can tell us a great deal about the effects of a new treatment and give sufficient credibility to make claims about a new idea, system, product or cultivar. However, plants are part of a biological system naturally prone to individual differences and biases, so understanding the experimental method and why it is used is a great tool for anyone wanting to carry out evaluations.
