QED+&+ED+Group+3

Shou-Bang, Leslie, Jenn, ** Joy ** ** Joy- purple **  SJ- green   Jen- blue   Lesley - red OR //italics in any color - having trouble holding the "red".// __Introduction__ Formally state the problem to be researched and briefly discuss why this question is important to the educational and research communities I've finally arrived. For the QED, we remove the control group, yes? Something on online discussions? Compare online and face to face or maybe even hybrid - which is more effective? Set up QED study. Sounds OK? Joy (Purple) Sounds good- I have articles on online discussions that I researched last semester. OK, agreed. Let me download the article and read it and see the adaptations we can do. Jen. I've skimmed through your article. Very difficult Math. Complex. May I counter propose another one. See how you feel about this Campbell article:
 * Group 3 (**
 * ) Please post your research proposal here. Each time you'd like to add something, simply click on "Edit." You may also use the "Discussion" area to communicate with each other.**

Suppose we pick an article (either Jen's or Joy's), what do we do next? Do we replicate it? Adapt it? I am a bit confused. Also, since Leslie hasn't shown up yet, shall we proceed ahead without her? SJ.

Hi- we need to come up with our own study. We can use those references to support our work and also use them in the reference information as supporting evidence. I think we should find a few more articles. I sent Leslie an e-mail to her personal e-mail address. I say we should just continue working on it as the deadline is approaching.

Okay. Now we need to find a topic. I posted in Discussion section on Friday a suggestion: "I am thinking of another one. How about purely online classes? It's easier to implement and design. We use survey to collect data from students. And the treatment is, say, an additional tiny module that's intended to help students learn online learning environment. One group/class receives this treatment, while the other does not. And then we see if the treatment has any effect or not on learning. Does it make sense?" Or, the treatment can be having students write reflective journals or not. Students can be randomly assigned to the control group and experiment group. Then we compare the outcome (through quizzes, like Jen's article, or through different measures.) - SJ

OK, can we design a simple experimental study. Pre test, post test - something like that? reflective journals will be more consistent with qualitative study. Shall we use surveys, post test and pre test and stop there? I think that will be sufficient. We need a treatment. So what are we trying to improve? Which variable specifically? It must be measurable. Maybe apply treatment to make people write more online. We design a treatment. The we measure performance before treatment. Apply treatment, then measure performance again. One experimental group, one control group. Sounds OK? The we use one survey to ask for perception of improvement. We can use the [|retrospective pre-post questionnaire].to measure before and after perceptions of treatment. Paired T-test. This is to find out:

"Did the participants experience a positive change in knowledge, awareness, confidence, and attitudes after the treatment?". We collect data using a survey. sounds OK? Joy

Were the rest of you able to access Joy's link for the retrospective pre-post questionnaire? Joy, I cannot access it - I keep getting a box to insert a new link. would you send this on bls to me? --Lesley

Here is the link Lee. Try it again: http://www.joe.org/joe/2003august/tt4.php

Treatment suggestions - just playing with a few. Let me know if you like any of them or suggest others: Treatment 1. The use of a longer and more specific personal introduction or "get to know you" activity early in the course with the intention of making students more comfortable with one another? Treatment 2: Each class member has a "buddy" in the class to bounce ideas off with and brainstorm with in order to feel more comfortable posting and interacting on-line as a member of the entire class? Treatment 3: The professor/teacher/educator sends individual (private) messages to each student 2 or three times per semester asking student for feedback on his/her online experiences with the class so far? (social and educational)

I like treatment # 3 and 1,

I can assess Joy's link. (Leslie, I think your problem with the links is that you tried to open the links in EDIT mode, that's why you'd be prompted to enter an URL. Before you hit the EDIT bottom, try again, they will work.) I like treatment #1. Thank you, Shou-bang. that was the problem. In terms of measurement, can we just use survey to understand student's satisfaction of the course? Our hypothesis can be that, the treatment, say, #1, can increase students satisfaction of the course. Yes- I think survey would be the best method. Do you think we should come up with just a few questions? (see under techniques)

10/11 - 2:00PM Hi everyone- I added some information below this AM. Please let me know what you think of it- Jen Looks good, Jen, thanks.

This is difficult, having preparatory comments and drafts mixed up. I am still not sure what we've come to agreement. (1) Have we decided what the treatment is? #1? (2) What is the dependent variable? What do we want to measure? Is it Student's Satisfaction? Student's learning performance (test scores)? Student's perception of online learning issues? (3) anything else?





Maybe we should use color-coded text to indicate changes we made on this page too. SJ (green)

Literature Review Normally, this is fairly extensive and details existing research that has already been done on the area of inquiry. For the purpose of this project, please omit the literature review.

Title:

Key Words: online, discussion, experimental design, purposive sampling, introduction exercises

__Method__ __- Methodology:__ If possible what methodological approach has been taken by previous researchers in this area and what approach do you intend to take?

I think this is where we need to tie in our studies/articles that we found.

__- Design__ This is where you discuss the type of study you’ll be undertaking. In this case, you'll be describing the QED or ED approach you have chosen and why it makes sense for your investigation.

In conducting this study, the experimental design method will be used to conduct this study (including random sampling, a control group and quantitative data analysis of pre-treatment and post-treatment surveys).

__- Population/Sample__ Who will be the subjects of your research? Will they be randomly assigned to groups or will they self-select? Give as much detail about the participants as possible The 50 participants in this study are a group of graduate students in the M. Ed Research Methods course at the [|University at Albany]. Graduate students are from the masters and doctorate programs in education.The study was conducted over a 15 week semester from January 2009 to May 2009. Out of the 50 participants, there were 32 females and 18 males. The participants ranged in age from 24 to 53 years old. The participants for the study were selected through __ random __ __purposive__ sampling from intact classes. 70% of the participants indicated that they have some experience in learning online and were familiar with activities associated with online learning like discussions, journaling, collaborative work, and peer-review of written assignments. The remaining 30% of participants have previously not participated in online course discussions.

Hi Jen, Thanks. But I wasn't sure about the random purposive sampling (underlined, see above). I think in ED randomization applies to the assignment of who gets the treatment and who don't. In the above paragraph, it's about how we chose these 50 participants. So purposive sampling should be fine. So then do we need to define more of why we selected the 50 participants that we did. I think by not explaining it, it may lead to biases and maybe decrease our validity.

Randomly assigned -

Adult learners Undergraduate and/or graduate (do you think we should stick to one or another?) Public or Private school? Age Range- should we have one? What year? Gender Previous online experiences? Yes or No ... Specific discipline area?

__- Data Collection/Analysis__ What are the important variables under consideration? How will you analyze your data and what steps can be taken to ensure as much validity as possible. The important variables under consideration are: frequency of a student's (written) participation in on-line class discussions; number of discussions initiated by a student, either in question or statement format; number of times a student responded to other students beyond the required number directed by the course instructor; student perceptions of their own levels of comfort in participating in online courses. Also, we could use the tracking system in Blackboard to obtain some more results such as the # of times or amount of time they spent in the discussions. Data collection will include: (Is this where we put some general information that Jen already posted below in techniques?) Pre- and post- treatment surveys for both experimental and control groups; counting of on-line responses from students both groups, including some types of responses (question vs. statement, addressed to other students or to instructor, required vs. non-required ).

__- Techniques__ Briefly describe how you will collect the relevant data. Will you need specific instruments? How will you ensure that you are collecting sufficient quantities of data?

Several data collection methods were used. In order to measure changes in knowledge and behaviour before and after the treatment, a retrospective pre-post survey, was used. Specifically, 30 questions were developed for the questionnaire in order to measure changes in constructs which included change in knowledge, awareness, confidence, and attitudes. According to Rockwell and Kohn (1999, in Davis, 2003), this tool is useful for evaluating the impact of ( intervention programs) by asking participants to report actual changes in behavior. The retrospective pre-post test was administered once before the treatment was administered and another time after the treatment. A paired t-test was used to analyze data in order to determine if the treatment had resulted in changes in behavior and knowledge.

Pre and post surveys were also used. Pre and Post surveys. Pre-surveys allow us to ask the demographic data such as their program, age range, previous experience with online discussion, if previous experience what types and was it solely hybrid or online,

Post surveys- is there where we would identify the learning activities and also measure if the online classroom/welcome area impacted theirparticipation? Something like: do graduate students perceive that the introduction activities and welcome module (presumably including pre-academic community building questions and discussions) make students feel comfortable to a) join in online discussions b) ask questions c) respond to a variety of their classmates (maybe even an estimate of how many different classmates they responded to) d) inspire them to participate in discussions well beyond the minimum requirement e) a feeling of familiarity, commeroderie (sp?), or friendship with other members of the class f) desire to take more online courses with the same instructor? g) desire to take more online courses in general?

Should we use a program like [|Qualtrics] to gather out results through the pre and post tests.

Sample of Questions: (maybe yes/no and Likert Scale) Age Range (years old) Question: 24 or younger, 25-35, 36 to 46, 47 to 57, 57 and older Do you have experience with online discussions in a classroom setting? If you answered yes to the previous question, was the course(s) hybrid, online, or a combination of both? How many online courses have you participated in? How many hybrid courses have you participated in?

[I am not good at measurement. Here's my thought. Since our main concern is whether the treatment causes the effect, so it more or less is up to what the dependent var is (what we want to measure). If we decide to look for content-knowledge, then we need a pretest and posttest for comparison and quantifying improvement. But if we decide to see student's perception of online learning, for instance, then a pre-survey and post-survey would be informative. If we decide to see student's satisfaction, then pre-survey would not make sense.] I guess the quantitative variable is the student's participation - we evaluate their perceptions on feeling at ease participating on-line pre- and post-treatment. I agree.

__- Ethical Considerations__ Are there ethical issues that may arise from this study? If so, how are they addressed and minimized? Discussion

[Because of my background, I can work on this section if that's okay with you guys. ]

(1) Students' learning outcome may be affected by the treatment. Some may benefit from it while other may not. That's a concern of fairness. (I don't think this is an issue, but will put it out there anyway.) [I think this is an issue. Since receiving the treatment or not will make the learning experience different, that means students of the same class are receiving different education, and parents would be concerned, and there we have an issue of fairness.]

(2) Students who have the resources to own their own computers and have 24/7 access to their courses MIGHT benefit more than students who can not afford a computer and are limited to university computer lab access hours and locations. (Also- with adult learners they could be working from a relatives computer, work computer, or library).

(3) IRB approval

(4) Consent of Instructor, Dean, etc.

__Potential Implications of our research__. If your hypothesis is correct, what impact will it have on the educational community? Are there any limitations of your study and what conclusion do you hope to be able to draw from this work?

(1) The practical implication is that online courses should incorporate Treatment into course design to improve student satisfaction. A correct hypothesis indicating that time spent by the instructor at the beginning of an online course for detailed introductions and online "community building" exercises will increase overall student online participation in discussion links, including initiating or extending conversations beyond the minimum course requirements. This, in turn, will increase student comfort level, confidence, and engagement in the learning process.

(2) Further study: determine the extent to which other variables related to student performance (knowledge, confidence, etc) can be improved by the Treatment. (This is a great idea, Shou-bang.) Further study:1. a qualitative analysis of the quality (not just quantity) of online student participation when this treatment is applied. 2. Studies comparing a variety of introduction approaches by instructors of on-line courses determine most effective styles.


 * __References__**

Please include citations used in constructing this project. It need not be extensive.

Campbell, M., Gibson, W., Halla, A., Richards, D., Callery, P. (2008). Online vs. face-to-face discussion in a web-based research methods course for postgraduate nursing students: A quasi-experimental study. International Journal of Nursing Studies, 45, 750–759. doi:10.1016/j.ijnurstu.2006.12.011.

Lin, S., & Chiu, C.-K. (2008). Incentive styles, asychronous online discussion, and vocational training. J. Educational Computing Research. 39(4) 363-377.

Stevens, G. L., & Lodl, K. A. (1999). Community coalitions: Identifying changes in coalition members as a result of training. Journal of Extension [On-line]. 37(2). In G, Davis (2003). Using a retrospective pre-post questionnaire to determine program impact. Extension Journa, 41(4). Retrieved from []

Should we include the Campbell article (see above) posted by Jen?