On this day...

Sending Charts from Google Forms

We're getting ready to launch a digital teaching and learning course for staff that is based on the TPACK framework. A few years ago, we started our 1:1 training by coaching teachers through using SAMR to design lessons which really use the technology in new ways. This time, we're having teachers do more metacognitive reflection on their own practice with TPACK as a model for thinking through content, pedagogy, and technology's impact on their practice.

Matt Kholer has collected several surveys which can be used to gauge each component of TPACK. We modified a 2009 instrument written by Kohler (and several others) into a Google Form for teachers to take before and after the program. For our course, we're only focusing on technology and pedagogy and coaching people through applying their content knowledge.

Our goal was to have teachers compelte the form and then receive an email with their survey results. Once the form was submitted, the sheet needed to average each domain, create a chart, generate an email, and fire it off with the chart included in the body of the email.

I started with Martin Hawksey's posts on email dashboards from Google Sheets using Apps Script. The main issue I ran into is that the radar chart we were hoping to use had to be rendered into the spreadsheet to get all the labels and formatting correct. Even calling the image directly from its URL was problematic. Specifically, if multiple people submitted at once, the charts could get combined in unexpeected ways, sending people wrong information. I looked into using Apps Script's LockServiceto limit execution, but honestly, I couldn't spend a whole lot of time on getting this running.

Martin suggested using quickchart.io to generate charts through an API endpoint. It's based on Chart.js, which I've used in the past for an action research summary.

This ends up working better because I don't need to read the sheet twice to generate a chart. I can take the processed data object right from the form responses, hit the API with some config in a JSON string and get an image which is appended into the email.

The Code

function processFormValues(row) {

    // The form structure doesn't change, so we can slice the values array directly.
    let obj = {
        "email": row[1],
        "tk": row.slice(2, 8),
        "pk": row.slice(8, 15),
        "tpk": row.slice(15, 21),
        "tpck": [row[row.length-1]],
    }
    return calculate(obj)
}

// calculate the average for the list of scores
function calculate(obj) {
    let avgObj = {}

    for(var key in obj) {
        if(key === "email") {
            avgObj["email"] = obj[key]
        } else {
            var length = obj[key].length;
            // Form responses are strings, so convert each to a number while calculating the average.
            var avg = (obj[key].reduce((prev, curr) => Number(prev) + Number(curr), 0)) / length;
            avgObj[key] = avg.toFixed(2) // limit the float
        }
    }
  return avgObj;
}

function getChart(chartData) {

    // Use chart.js to avoid race conditions building charts in the sheet.
    const baseUrl = `https://quickchart.io/chart?bkg=white&c=`
    const params = {
        "type": "horizontalBar",
        "data": {
            "labels": ["tk","pk","tpk","tpck"],
            "datasets": [
                {
                    "label": "Self Assessment",
                    "backgroundColor": "rgba(54, 162, 235, 0.5)",
                    "borderColor": "rgb(54, 162, 235)",
                    "data": [ chartData[1], chartData[2], chartData[3], chartData[4] ]
                }
            ]
        },
        "options": {
            "plugins": {
                "datalabels": {
                    "display": true,
                    "anchor": "end",
                    "align": "end",
                    "color": "#000",
                }
            },
            "elements": {
                "rectangle": {
                    "borderWidth": 2
                },
            },
            "responsive": true,
            "scales": {
                "xAxes": [{
                    "ticks": {
                        "display": true,
                        "beginAtZero": true,
                        "suggestedMax": 6,
                        "stepSize": 1,
                    },
                "gridLines": {
                    "display": true,
                }
            }],
            "yAxes": [{
                "gridLines": {
                    "display": false,
                }
            }]
        }
    }
}

    try {
        let blob = UrlFetchApp.fetch(baseUrl + encodeURIComponent(JSON.stringify(params))).getAs('image/png');
        return {'url': baseUrl + encodeURIComponent(JSON.stringify(params)), 'blob': blob};
    } catch(e) {
        return e.message
    }
}

// Add an installable trigger to run on each form submit
function onFormSubmit(e) {
    const ss = SpreadsheetApp.getActiveSpreadsheet();
    const calcs = ss.getSheetByName('calculations');
    const headers = calcs.getRange(1, 1, 1, calcs.getLastColumn()).getValues()[0];

    try {
        let scores = processFormValues(e.values);

        // Save the averaged data to a sheet in case something goes wrong
        var writeRow = []
        for(var i=0; i<headers.length; i++) {
            writeRow.push(scores[headers[i]]);
        }

        // Assign to a variable because we'll write to the sheet later.
        let nextRange = calcs.getRange(calcs.getLastRow()+1, 1, 1, writeRow.length).setValues([ writeRow ])

        // Get the chart
        let chart = getChart(writeRow)

        // if there was good data then send the email
        if(chart) {
            let emailImages = {};
            emailImages['chart'] = chart['blob'];

            let emailBody = `...your email template...`;

            // append the image as a blob
            emailBody += `<p align='center'><img src='cid:${emailImages["chart"]}'></p>`;

            MailApp.sendEmail({
                to: writeRow[0],
                subject: 'Your survey results!',
                htmlBody: emailBody,
                inlineImages: emailImages
            })

            // Set the chart URL in a sheet in case we need it again
            calcs.getRange(nextRange.getRow(), 6).setValue(chart['url']);
        }
    } catch(err) {
        calcs.getRange(calcs.getLastRow()+1, 1).setValue(err)
        throw new Error(err) // put it in the console
    }
}

The result

As form submissions come in, the sheet generates the average for each category and then uses those values to send a request to the quickchart API. The image blob is captured in an object and sent inline to the user who submitted. It runs in 7-8 seconds consistently, with the lag coming from sending the email.

https://blog.ohheybrian.com/wp-content/uploads/2020/10/2020-10-21_09-59-42.png

Our email template includes an explainer on what each domain is meant to describe so participants can think through their survey before coming to the live session. We'll use the same form at the end so they can compare their results as part of the final reflection.

Displaying Aggregate Data in Google Sheets

Our students are required to have their student ID on when they come into the building each day. This is mainly for security, but their ID also allows cafeteria, library, and other office staff to scan their ID for various clerical tasks.

When a student *doesn’t* have their ID, they’re issued a temporary wristband so faculty and staff know they’ve checked in for the day and that their record has been updated. The update process involved a paper copy made at a cart with wristbands being passed to a secretary who would then look up the student and add it to a spreadsheet before adding it to our SIS. Dang.

The head secretary (Deb) and I worked out a Google Sheet that does all of that (minus the SIS portion, but that’s coming soon).

I’ll outline the functions below, but you can [grab a copy of a blank sheet to play with](https://docs.google.com/spreadsheets/d/1MHrQVM0tPql8gx2tKfFlJwc5teNyDx8-LJ1FRulKFv0/copy) if you’d like to tinker or improve.

Raw info in…

At it’s simplest, the staff member at the check in kiosk has to enter the date and the student’s (supposed) ID number. This sheet populates the rest of the workbook.

…rich info out.

Once the date and ID have been put in, some behind-the-scenes magic happens. This isn’t ideal, but for right now, we’re using a hidden sheet with a master record of student ID’s and their associated name. I would like to make this a web-based API call, but it isn’t possible (not a technical issue…internal) right now.

  1. A staff member enters the student ID into the `raw` data sheet. The ID is cross-referenced in the `lookup` sheet and the name is displayed. This is to avoid students giving the incorrect ID.
  2. The `noID` sheet gathers unique IDs from the `raw` list so we can aggregate the number of times they’ve come in with an ID. The student name is again pulled from the `lookup` list.
  3. A simple `countif` formula counts the number of times an ID shows up in the list and aggregates. Those cells also change color using conditional formatting depending on the number of entries.
  4. Consequence data is filled in automatically depending on the number of entries a student has.

Disaggregating the Aggregate

For reporting, it’s helpful to know the actual incident dates. Going through each row, one by one, looking for a student name (even using the Find tool) can take a while, especially in large sheets. So, we turn to Google Apps Script to do it for us.

The comments in the script should give you an idea of what’s happening, but if you didn’t follow, here’s the general idea:

– Select an ID you want to look up.

– The function looks for that ID in the `raw` sheet. All matched dates are stored in an array.

– Once the loop is finished, the full array is passed to a function which adds the note to the selected cell.

– You can now see the individual dates a student had no ID by hovering over their ID.

It’s done case-by-case and only when the script is selected in the menu so your sheet isn’t trying to constantly update. The idea being that you only use the script when you need to know the dates a student didn’t have an ID when they came in.

What do you think? Leave a comment below, or better yet – make it better and write up your version with a link back here.


[flickr photo by Owen P](https://flickr.com/photos/owenphilipson/26858459235) shared under a Creative Commons (BY) license