Quant: Validity and Reliability

the construction process of a survey that would ensure a valid & reliable assessment instrument

Most flaws in research methodology exist because the validity and reliability weren’t established (Gall, Gall, & Borg, 2006). Thus, it is important to ensure a valid and reliable assessment instrument.  So, in using any existing survey as an assessment instrument, one should report the instrument’s: development, items, scales, reports on reliability, and reports on validity through past uses (Creswell, 2014; Joyner, 2012).  Permission must be secured for using any instrument and placed in the appendix (Joyner, 2012).    The validity of the assessment instrument is key to drawing meaningful and useful statistical inferences (Creswell, 2014). Creswell (2014), stated that there are multiple types of validity that can exist in the instruments: content validity (measuring what we want), predictive or concurrent validity (measurements aligned with other results), construct validity (measuring constructs or concepts).  Establishing validity in the assessment instrument helps ensure that it’s the best instrument to use for the right situation.  Reliability in assessments instruments is when authors report that the assessment instrument has internal consistency and have been tested multiple times to ensure stable results every single time (Creswell, 2014).

Unfortunately, picking up an assessment instrument that doesn’t match the content exactly will not benefit anyone, nor will the results be accepted by the greater community.  Modifying an assessment instrument that doesn’t quite match completely, can damage the reliability of this new version of the instrument, and it can take huge amounts of time to establish validity and reliability on this new version of the instrument (Creswell, 2014).  Also creating a brand new assessment instrument would mean extensive pilot studies and tests, along with an explanation of how it was developed to help establish the instrument’s validity and reliability (Joyner, 2012).

Selecting a target group for the administration of the survey

Through sampling of a population and using a valid and reliable survey instrument for assessment, attitudes and opinions about a population could be correctly inferred from the sample (Creswell, 2014).  Thus, not only is validity and reliability important but selecting the right target group for the survey is key.  A targeted group for this survey means that the population in which information will be inferred from must be stratified, which means that the characters of the population are known ahead of time (Creswell, 2014; Gall et al. 2006). From this stratified population, is where a random sampling of participants should be selected from, to ensure that statistical inference could be made for that population (Gall et al., 2006). Sometimes a survey instrument doesn’t fit those in the target group. Thus it would not produce valid nor reliable inferences for the targeted population. One must select a targeted population and determine the size of that stratified population (Creswell, 2014).  Finally, one must consider the sample size of the targeted group.

Administrative procedure to maximize the consistency of the survey

Once a stratified population and a random sample from that population have been carefully selected, there is a need to maximize the consistency of the survey.  Thus, researchers must take into account the availability of sampling, through either mail, email, website, or other survey tools like SurveyMonkey.com are ways to gather data (Creswell 2014). However, mail has a low rate of return (Miller, n.d.), so face-to-face methods or online the use of online providers may be the best bet to maximize the consistency of the survey.

References

Creswell, J. W. (2014) Research design: Qualitative, quantitative and mixed method approaches (4^th ed.). California, SAGE Publications, Inc. VitalBook file.

Gall, M. D., Joyce Gall, Walter Borg. Educational Research: An Introduction (8^th ed.). Pearson Learning Solutions. VitalBook file.

Joyner, R. L. (2012) Writing the Winning Thesis or Dissertation: A Step-by-Step Guide (3^rd ed.). Corwin. VitalBook file.

Miller, R. (n.d.). Week 5: Research study construction. [Video file]. Retrieved from http://breeze.careeredonline.com/p8v1ruos1j1/?launcher=false&fcsContent=true&pbMode=normal

Quant: Getting Lost in the Numbers

In quantitative research, data that is collected is numerical in nature. Rarely is every member of the population studied, and instead a sample from that population is randomly taken to represent that population for analysis in quantitative research (Gall, Gall, & Borg 2006). At the end of the day, the insights gained from this type of research should be impersonal, objective, and generalizable.  To generalize the results of the research the insights gained from a sample of data needs to use the correct mathematical procedures for using probabilities and information, statistical inference (Gall et al., 2006).  Gall et al. (2006), stated that statistical inference is what dictates the order of procedures, for instance, a hypothesis and a null hypothesis must be defined before a statistical significance level, which also has to be defined before calculating a z or t statistic value.

Essentially, a statistical inference allows for quantitative researchers to make inferences about a population.  A population, where researchers must remember where that data was generated and collected from during quantitative research process.  However, it is easy to get lost in the numbers during quantitative research, thus here is a list of some of the ways to keep the focus on the people and organizations when research deal with the numbers that represent their population: To design a quantitative research project, researchers must understand the purpose and rationale of their own research designs and their research methods (Creswell, 2014).  Knowing the purpose and rationale can help the development of a research question(s) and hypothesis.  With a clear research question and hypothesis can a researcher to design and review their data collection from people, organizations, or instruments.  It is when focusing on the methods section that researchers can keep their focus on the people and organizations by identifying the population, consideration of a stratified population before sampling, sampling design and procedures, selection process for the individuals, which variables to study (their name, how they relate to the research question, and collection description) (Creswell, 2014).

• The numerical data used in the quantitative research was generated and collected from people, a social group, an organizational entity, or an instrument. The numerical value alone does not have any meaning nor value to the research. But, when the numerical value is paired with contextual information, then it provides researchers a wealth of information to conduct their statistical analysis on the data (Ahlemeyer-Stubbe, & Coleman, 2014; Miller, n.d.a.).
• Remember each data point, row or column represents a person, group, or thing with all its features and bugs. It would be wise to create a metadata file that describes the data points variables to help keep the focus on the people and organizations.  In SPSS, the metadata section is called the “Variable View”, and each person is represented as an entity or row of data in the “Data View” (Field, 2013; Miller, n.d.b.).
• Data sets are never neutral and theory-free data repositories but require researchers to interpret that data through their personal lenses (Crawford, Miltner, & Gray, 2014). One must gather and analyze data ethically to avoid social and legal concerns. Thus, the researcher must be aware of how their analysis of the data can be used to cause harm to others or help facilitate discriminate against disenfranchised groups of people (Robinson, 2015).

References:

• Ahlemeyer-Stubbe, A., & Coleman S. (2014). A practical guide to data mining for business and industry. UK, Wiley-Blackwell. VitalBook file.
• Crawford, K., Miltner, K., & Gray, M. L. (2014). Critiquing Big Data : Politics , Ethics , Epistemology Special Section Introduction. International Journal of Communication, 8, 1663–1672.
• Creswell, J. W. (2014) Research design: Qualitative, quantitative and mixed method approaches (4^th ed.). California, SAGE Publications, Inc. VitalBook file.
• Field, A. (2013) Discovering Statistics Using IBM SPSS Statistics (4^th ed.). UK: Sage Publications Ltd. VitalBook file.
• Gall, M. D., Gall, J., & Borg W. (2006). Educational research: An introduction (8^th ed.). Pearson Learning Solutions. VitalBook file.
• Miller, R. (n.d.a.). Week 1: Central tendency [Video file]. Retrieved from http://breeze.careeredonline.com/p9fynztexn6/?launcher=false&fcsContent=true&pbMode=normal
• Miller, R. (n.d.b.). Week 2: All about SPSS. [Video file]. Retrieved from http://breeze.careeredonline.com/p99kywtldbw/?launcher=false&fcsContent=true&pbMode=normal
• Robinson, S. C. (2015). The good, the bad, and the ugly: Applying Rawlsian ethics in data mining marketing. Journal of Mass Media Ethics, 30(1), 19–30. http://doi.org/10.1080/08900523.2014.985297