Learning analytics (defined here) is a growing field within both educational and institutional research. By gathering and analyzing user data, we can work to answer a variety of learning and behavioral questions. This week, we ask Jing Qi of Dartmouth EdTech: What does a day look like for Dartmouth students participating in Canvas? This is what we found!
ETC: How is this user data harvested? Who was included?
JQ: We had to define what user data we needed to collect and what definition we would use for participation.
In this case, we harvested user data for students enrolled in at least one undergraduate Arts and Sciences course for the spring term. We focused on a 24 hour period on April 20th, 2015 beginning at 4:00 AM.
Canvas automatically collects a variety of user behavior data that we can harvest via the Canvas API. We have to know how to identify and access the data of interest for our inquiry purposes. Canvas has a working definition for participation we worked off of in this case.
At that point, we use a combination of harvesting and decoding of the data. Part of my expertise is in knowing the data that is readily available for collection. For the purposes of this question, we looked at specific participation data in the 24 hour period. This included:
Assignment submissions - the assignments that were actually turned in via the online uploader.
Collaborations - we collected information on collaborations started or joined using the Conferencing and Collaboration tools.
Discussion posts - including both posts created and comments on discussions in Canvas.
Page edits - we collected information on page edits via the Canvas wiki pages tool.
Quiz submissions - we looked specifically at quizzes submitted. We did not include quizzes started without being submitted.
ETC: What do you see when you look at the data? What is missing?
JQ: When I look at the data, I immediately think about patterns. One of my first thoughts was about the limitations of the information we had in this case. We were looking narrowly at one 24 hour period with one specific set of user generated data.
I was really intrigued by the spikes occurring in the early evening and really early morning hours. I would be really curious to correlate whether the student behavior is associated directly with imposed due dates. I would be curious how much of this is explained by student study habits and working patterns.
I was also surprised by the amount of discussion activity. We know anecdotally that there are a variety of graded and ungraded discussion assignments in courses. Students seem to be fairly active during certain time frames.
One of my favorite things is that with every investigation with learning analytics, new doors are opened to other research questions and inquiry paths.
ETC: What can Dartmouth faculty and collaborators do to get involved with learning analytics?
JQ: It’s really important to think about the foundational questions you are interested in pursuing. Faculty with questions of interest can email me if they would like to talk more (firstname.lastname@example.org). I can certainly help them to think about the types of data that is readily available to them. They could also talk with an Instructional Designer about their course design and assessment strategies.
Do you have questions about learning analytics? Tell us what you think about in the comments and on social media!