Tag Archives: validity

Reducing the Items in the Research Instrument

There are instances that you need to reduce the items in your research instrument. If the items in the instrument that you have prepared are too many, chances are, your respondents would not read it thoroughly and eventually give you a poor result. Let’s admit that nowadays, respondents don’t like to answer too many questions. For them, it is a waste of time. With this in mind, how can you lessen the number of items in your questionnaire? Here are your options:

1. Validate the contents of your research instrument.

Aside from measuring the validity of the instrument, content validity can help you decide which items must be deleted, thus reducing the items in your questionnaire. Content validity can be done by the experts in that field that you are trying to investigate. But who are considered “experts?” These are the people who have doctorate degrees in that field and have practiced their profession for many years. Usually, a minimum of three persons must take a look at your instruments.

However, if your paper is towards fulfilling a master’s degree requirement and there is no one who has a PhD in master studentthat area, those who have master’s degree can be taken as experts. If master’s degree holders are still unavailable, the panel of examiners may allow you to avail the help of those who have been practicing their profession for not less than ten years as “experts” in that field.

These people are knowledgeable as to the depth of the contents of the subject area that you want to investigate in and must be adept as to which items should be included to determine a particular variable (construct) and its sub-variable (sub-construct) in your study. They can also tell you which of the items from your questionnaire could be deleted.

2. Do factor analysis for data reduction.

If you choose to do a factor analysis, this will help you determine the construct validity of your instrument and help you decide, with the help of your statistician, which items are to be culled out from your instrument. To know more on this, see What are the Psychometric Properties of a Research Instrument?

3. Have both content and construct validity.

Having the two types of validity, I believe, will make your instrument better. As a researcher, you are the one to decide or with the help of your research adviser, which items must be deleted using the results of content and construct validity.

Generally, instruments which underwent the content and face validity are much shorter and can give more accurate results than one which did not. Thus, this gives you more confidence in the interpretation of your data.

What are the Psychometric Properties of a Research Instrument?

Here is a differentiation of reliability and validity as applied to the preparation of research instruments. 

One of the most difficult parts in research writing is when the instrument’s psychometric properties are scrutinized or questioned by your panel of examiners. Psychometric properties may sound new to you, but they are not actually new.

In simple words, psychometric properties refer to the reliability and validity of the instrument. So, what is the difference between the two?

Reliability refers to the consistency while validity refers to the test results’ accuracy. An instrument should accurately and dependably measure what it ought to measure. Its reliability can help you have a valid assessment; its validity can make you confident in making a prediction.

Instrument’s Reliability

How can you say that your instrument is reliable? Although there are many types of reliability tests, what is more usually looked at is the internal consistency of the test. When presenting the results of your research, your panel of examiners might look for the results of the Cronbach’s alpha or the Kuder-Richardson Formula 20 computations. If you cannot do the analysis by yourself, you may ask a statistician to help you process and analyze data using a reliable statistical software application.

correlation

But if your intention is to determine the inter-correlations of the items in the instrument and if these items measure the same construct, Cronbach’s alpha is suggested. According to David Kingsbury, a construct is the behavior or outcome a researcher seeks to measure in the study. This is often revealed by the independent variable.

When the inter-correlations of the items increase, the Cronbach’s alpha generally increases as well. The table below shows the range of values of Cronbach’s alpha and the corresponding descriptions on internal consistency.

Cronbachs alpha

(Note: The description is not officially cited and taken only from Wikipedia, but you may confer with your statistician and your panel of examiners. If the value of alpha is less than .05, the items are considered poor and must be omitted).

Instrument’s Validity

There are many types of validity measures. One of the most commonly used is the construct validity. Thus, the construct or the independent variable must be accurately defined.

To illustrate, if the independent variable is the school principals’ leadership style, the sub-scales of that construct are the types of leadership style such as authoritative, delegative and participative.

The construct validity would determine if the items being used in the instrument have good validity measures using factor analysis and each sub-scale has a good inter-item correlation using Bivariate Correlation. The items are considered good if the p-value is less than 0.05.

References:

1. Kingsbury, D. (2012). How to validate a research instrument. Retrieved October 16, 2013, from http://www.ehow.com/how_2277596_validate-research-instrument.html

2. Grindstaff, T. (n.d.). The reliability & validity of psychological tests. Retrieved October 16, 2013, from http://www.ehow.com/facts_7282618_reliability-validity-psychological-tests.html

3. Renata, R. (2013). The real difference between reliability and validity. http://www.ehow.com/info_8481668_real-difference-between-reliability-validity.html

4. Cronbach’s alpha. Retrieved October 17, 2013, from http://en.wikipedia.org/wiki/Cronbach%27s_alpha

© 2013 October 17 M. G. Alvior

Five Weaknesses of the Survey Method

When not designed properly, data obtained using the survey method is next to useless. Find out why.

While a lot of information can be gained from surveys as it is easy to get large samples using the method, results should be taken with caution because of its inherent weaknesses. This is the reason data analysts do not treat data obtained from surveys in the same way as those obtained using other means, that is, those that do not rely on people’s opinions or subjective judgment.

So what are the weaknesses of the survey method? Below is a list with brief explanations.

Weaknesses of the Survey Method

1. Respondents protect their interest

There is really no guarantee about the truthfulness of the respondent’s answers. When dealing with sensitive or controversial issues, there is a tendency among interviewees to avoid answers that may be detrimental to their interests. They may not even answer the question at all.

2. Attitude is different from behavior

As the famous expression goes: “Do what I say, not what I do.” A considerable number of research on the relationship between attitude and behavior has demonstrated that there is no correlation between what people say they would do than their actual behavior.

In a recent study in the United Kingdom, for example, 99% of people interviewed said they had washed their hands after using the toilet. The truth is, as revealed by electronic recording devices, only 32% of the men and 64% of the women actually did it.[1]

People know what is expected of them, pretend that they do it when actually they don’t.

3. Responses are taken lightly

Some respondents are not really predisposed to answering questions especially if it involves answering a series of long questions. Similarly, the interviewer may not be very keen about the data collection.

asking questions

4. Poor memory among respondents

Interviewees tend to forget past events, places, or experiences. Their answers, therefore, are mainly estimates or rough approximations as their memory would allow them to recall those things.

Would you rely on information recalled 20 years ago by a 60-year-old respondent? Of course not.

5. Answers can be manipulated

Some interviewees answer the questionnaires to gain a favor. People answer what they believe the interviewer wants to hear. This threatens the validity of the answer.

What Then Should Be Done?

To get the most from using the survey method, particular attention must be given to the design of the questionnaires,selection of samples, and administration. For greater reliability and validity, other methodologies are applied to confirm the findings of a survey (see Triangulation).

Reference

1. Winterman, D. (2012, October 15). Handwashing: Why are the British so bad at washing their hands? Retrieved September 2, 2013, from http://www.bbc.co.uk/news/magazine-19834975

© 2013 September 4 P. A. Regoniel

10 Tips on How to Discern Fact From Fiction

When presented with information, how do you assess its validity or reliability? Can you distinguish fact from fiction? Here are 10 tips to ponder.

I saw a film titled Primeval last week and I can’t help but get amused of the way the behavior of the large crocodile named Gustave is being portrayed. During the climax, Steven (played by Orlando Jones), cameraman of the news team that tried to capture Gustave, runs with all his might in his bid to outrun the literally galloping crocodile. Gustave, at the end of the film, was also trying to get inside the vehicle, ferociously snapping away at the passengers. So dramatic.

I had such a good laugh because I know crocodiles cannot sustain long runs on land. They are not designed to do so. Crocodiles suddenly lunge when opportunity comes to catch their prey. Thus, they are called opportunistic predators. The attack usually happens at the edge of the water.

Crocodiles cannot sustain long runs because lactic acid builds up easily in its muscles. I had readings and personal encounters on this fact as I once worked in a crocodile conservation facility as an ecologist. If the crocodile does not rest (just like the way you see lizards do it), it will most probably die due to too much strenuous activity[1]. This is characteristic of cold-blooded animals. According to a recent study from the University of Adelaide, if dinosaurs were cold-blooded, they would not have dominated the world for millions of years [2].

For this reason, I thought of writing up the following tips, to help discern fact from fiction, although I would say this is a tall order. In reality, it is difficult to see the truth when bombarded with a lot of conflicting information. These tips, however, can prove handy when reading material from the web and sort out reliable information from trash in view of enhancing your literature review.

10 Tips to Discern Fact from Fiction

1. Educate yourself.

If you have a good background of the subject, nobody can fool you into believing something that is grossly absurd. Imagine the amount of time you spend to educate yourself in schools. It is a long process, but it will help relieve your ignorance about many things.

2.  If things are too exaggerated or too good to be true, dismiss it.

Have you ever seen a film where the human characters fly and jump high places whenever they fight ? If these people exist, then this world will be different. With training, some people will no longer need a ladder or a car for that matter. With just a few hops, they will be able to reach their destinations.

3. Always question what is being presented to you.

discernfactfromfiction

Don’t take things as they are. If there is doubt in your mind regarding things, ask. Through asking, you will verify the existence of an object, fact, or event.

Can you distinguish dreams from reality? Ask yourself while dreaming. Chances are, you will not be able to do so.

4. Verify the sources of information.

Find out where your information comes from. Is it coming from a verified source or is it just someone’s opinion? This is the very reason researchers have always to write their sources or references to enable the reader or critic of his paper to verify sources for reliability.

Science builds up its foundation on facts. If the foundation is weak, everything built upon it collapses once it gives way. A theory is built out of many tests of hypotheses.

Verifying sources of information becomes very important especially on those occasions where something really important crops up. Say, a total cure for cancer has been developed or discovered.

Nobody in his right mind will believe a tricky quack doctor’s recipe who has had a track record of 49/50 (49 died out of 50). Also, a scientist’s word is more believable than a politician’s when it comes to new discoveries.

5. Get rid of your biases.

If you have your prejudices, then your objective judgement is clouded. Take things with a grain of salt. Don’t incorporate your emotions and your personal biases. You can avoid this by applying the principle of triangulation.

6. Assess how well information is presented.

When reading information online, which one would you believe – one that is poorly written full of grammatical errors or one that is professionally written? Of course, you will go for the latter.

Presentation matters. If something is carefully done, chances are, there is more truth to it.

7. Vary your perspective.

Observe things using different perspectives. Delay your judgment when the facts are not adequate.

Remember the story of the six blind men and the elephant (see the video below if you are not familiar with this story)? Each one of them has a different view of what an elephant is because they relied on only one observation. The point is: don’t confine yourself to just one observation.

[youtube=http://www.youtube.com/watch?v=iBqgr5xZLz0&rel=0]

8. Think beyond the box.

Explore what other information you can find about things around you. Thinking beyond the box means that you are unconventional. Don’t take things as they are. Ponder them and take action to verify the truth based on your observation.

The newly discovered mammal named olinguito would have been overlooked had one scientist dismissed his observation. Read about this discovery here.

9. Don’t make decisions right away.

Many commit blunders as a result of wrong decisions. Blunder is a common term used in playing chess. Once you make the decision, you cannot retract it. The die is cast.

While too much procrastination may be bad for you, being impulsive is also destructive. University of San Diego professor Frank Partnoy says the key to success is waiting for the last possible moment to make a decision[3].

10. Watch out for opinionated statements. 

Don’t believe something without a basis. If you have 100 people, you are bound to have 100 opinions. Arguments should be substantiated by facts or evidences.

The whole point of the matter discussed here is that it pays if you follow certain guidelines in evaluating information presented to you. Remember Alexander Pope’s famous quotation “Fools rush in where angels fear to tread.”

References

1. Queensland Government. (2013, April 24). Crocodiles. Retrieved from August 17, 2013, from http://www.qld.gov.au/environment/plants-animals/animals/crocodiles/

2. Outred, J. (2013, July 24). Cold-blooded dinos would have been ‘too weak’. Retrieved from August 17, 2013, from http://www.australiangeographic.com.au/journal/cold-blooded-dinosaurs-would-have-been-too-weak.htm

3. Gambino, M. (2012, July 13).  Why Procrastination is good for you. Retrieved August 17, 2013, from http://www.smithsonianmag.com/science-nature/Why-Procrastination-is-Good-for-You-162358476.html

© 2013 August 18 P. A. Regoniel