I’m teaching a couple of classes, Introduction to R and Introduction to SAS, and I’m finding that students will turn in homework a variety of different ways. I’m fine with this up to a point, but I think that I should encourage a simple uniform approach, because out in the real world, your boss or your clients will not appreciate a haphazard and disorganized approach. Here’s a suggested format for homework assignments that will (hopefully) get you in the practice of turning into things in an organized fashion.

Here are some guidelines for submission of your homework. Do not follow these guidelines slavishly, and if you have a good reason to ignore one of these recommendations, you will not be penalized. Try, however, to follow these guidelines as best you can. In the real world, you will find that your boss and your clients will appreciate an organized and consistent format.

Every assignment that you turn in should have a report of one page or less. The report is followed by tables, figures, and appendices.

Write your report in plain English with no formulas, no jargon, no computer code, and no raw output. Include the verbatim text of the homework assignment as part of your report, but use a style such as bold, italic, indentation, etc. so it is clear what you have written and what you have copied from the homework assignment.

Your report should have a header with your name, the name of the class, and the name of the homework assignment, and a date.

Your report should be short. Normally one page is sufficient, and for some assignments, you may need as little as a couple of sentences.

If you have graphs, they should be numbered and appear one per page with a brief descriptive title. You can put two or more graphs on the same page, but only if your intent is to compare or contrast those graphs in your report. The interpretation of your graph belongs in the main section and not with the individual graphs, with the possible exception of a brief title or a few labels on the axes or in the graph itself. If a graph that does not warrant a comment in your report, put it in the appendix or (better yet) leave it out entirely.

Each table should numbered and appear one per page with a brief descriptive title. With very rare exceptions, no table should take more than a single page. You can put two or more closely related tables on a single page, but only if your intent is to compare or contrast those tables in your main section. The interpretation of your tables belongs in the main section and not with the individual tables, with the possible exception of a brief title or a few footnotes. If a table does not warrant a comment in your report, put it in the appendix or leave it out entirely.

If you do not know how to interpret a graph or table that you generated, please post a question  in the trouble shooting section of the discussion board.

Your appendix will consist of the data dictionary for the raw data that you used and a changelog file if you made any changes to the raw data. You do not need a data dictionary for any files that you create as part of your homework assignment.

Also include the program code and the unedited computer output as separate appendices. For SAS, you should also include the log as a separate appendix. For R, include your code as a separate appendix, even if you are using R Markdown. For SPSS, make sure that you include the generated syntax with the output, but you would normally not have any code in SPSS if you are using the menu system.

If you use multiple programs to complete your assignment, you can use multiple appendices, but take care so the number of appendices does not become excessively large.

Do not submit any code, log, or output that has error messages in them. If you are getting an error message that you cannot fix, please post a question in the troubleshooting section of the discussion board.

If your code produces warnings, explain what those warnings mean and why it is safe in your particular context to ignore them.

If your output includes a printout of your raw data and/or your modified data, please print only the first ten rows and the first five columns of data. If clarity mandates a larger printout, you can exceed these limits but try your very hardest to keep to a single page for each data set. Please do not print out any intermediate data sets.

Each appendix should have a descriptive title.

Combine the report, tables, graphs, and appendices into a single file. I prefer PDF format, but will take html, doc/docx, or ppt/pptx files. Other formats may be okay, but ask me first. If you have trouble combining your files, please post a question in the troubleshooting section of the discussion board.

I will try to create some simple examples of what a homework submission should look like.

I’m teaching an online workshop for The Analysis Factor on survival analysis. It’s not announced yet, and I have a LOT of work to do before it is ready. One thing that will save me time is that I am taking many of my examples from the excellent textbook, Applied Survival Analysis Second Edition. One nice perk of this book is that the helpful folks at UCLA have taken every textbook example, and written up code (with comments!) to reproduce the book’s results. With the exception of a few advanced methods in later chapters, where only one or two software packages have the right capability, the code is written in parallel in R, SAS, SPSS, and Stata. They also have links to the raw data at the publishers website, and datasets stored in SAS format and SPSS format. How nice! Browse around and you’ll find software code for all the examples in other popular statistics textbooks as well.

Warning! The R examples look like they are from the first edition, not the second edition. A small nitpick for an otherwise very nice resource.

UCLA Institute for Digital Research and Education. Textbook Examples Applied Survival Analysis. Second edition by David W. Hosmer, Jr., Stanley Lemeshow, and Susanne May. Available at https://stats.idre.ucla.edu/other/examples/asa2/.

I’ve gotten some helpful feedback that I need to encourage more interactions among students in the on-line classes, Introduction to R, Introduction to SPSS, and Introduction to SAS. No just interactions of the students with the teacher, but interactions between the students.

In many online classes this is done by encouraging online discussion of the material in the class. This is not so easy, however, for these three classes. I can just imagine myself posting the following on Blackboard. “Tell me what you think about the read.csv function in R.”

There are a couple of ways, however, that make sense for technical classes like these.

An obvious strategy is to encourage students to comment on anything they find confusing in the videotaped lectures. For example, as student might say

“I got a strange error message when I tried to import the dataset with “read.csv(file=fn, header=TRUE)”

When you see this, rather than answer it right away, ask

“Did anyone else have this problem?”

Then wait a bit. If you’re lucky, another student might chime in and say

“I had that problem also, but when I stripped the first line from the file, and changed the option to “header=FALSE” it worked just fine. Except, I lost the variable names for each of the columns of data.”

Then another student might chime in and say

“I got the variable names back by cut-and paste of the first line into the R program itself. I had to reformat a bit, but it worked nicely.”

That’s the theory, anyway. It mimics the situation in a class where one person asks a question, and another student answers the question before the instructor can (which is ideal from the instructor’s perspective).

Now, it is hard to get students to offer commentary, either online or in a live class, but that’s something that you need to encourage if you want to be a good teacher. I must confess that I am not nearly as good at this as I should be. I tend to dominate any classroom discussion, to the detriment of myself and my students. But I’m working on it.

Another strategy for encouraging student interactions is to require peer-grading of assignments. These are pass/fail classes, so I’m not interested in a grade per se. What I want is some helpful commentary about the good things and bad things in another student’s assignment.

Again this is tricky for a programming class. Someone shares the code they used and the output that the code produced, and what is there to comment on other than the trivial comment that it worked.

One thing that you can get one students to critique about another student’s work is the quality of the documentation. One thing these classes need to emphasize more is placing a greater emphasis on documentation.

So a student might submit a project where they imported a text file, created value labels, and recoded obvious outliers as missing values. The first student submits their code and output, but also their data dictionary and their changelog. The second student provided feedback by answering the questions

1. Did you understand the information in the data dictionary?
2. is there any information that you wanted to see in the data dictionary that wasn’t there?

In order for peer evaluation to work at its best, you need to make sure that the second student is not working on the exact same data set that the first student is working on. It’s easy to overlook a poorly documented data dictionary for a data set that you’re already intimately familiar.

Fortunately, there are so many open source data sets out there that it won’t be hard to give each student a different data set to work on.

A second thing that one student can critique about another student’s work is the quality of the interpretation of the output. This requires a change in emphasis again. These classes do ask for interpretations when appropriate, but when the output is just a two by two crosstabulation like

       Dead Alive
Female  154   308
Male    709   142

there’s only so much you can say. So these classes need to incorporate things like a Fisher’s Exact Test for any two by two crosstabulation, so that students can say that males have a statistically significant increase in mortality in this data set compared to females.

I don’t want to go too deeply into statistical tests in these classes. My goal is to teach coding, not statistics. But adding a few more tests and confidence intervals, very simple ones, will allow for more interesting interpretation of the output.

So assignments should ask for the output and the code (for R and SAS) and the log (for SAS). But it should also ask for a brief report (no more than one page) interpreting the results. When the first student submits the report, ask the second student to address the following questions

1. Did you understand the interpretation provided?
2. Was it consistent with the raw output?
3. Was there anything else you wanted to see in the interpretation?

I have helped develop and have taught (along with other faculty in our department) three one credit hour pass/fail classes: Introduction to R, Introduction to SPSS, and Introduction to SAS. These classes were developed back in 2014-2015 and they are in need of some serious updates. I will try to outline some of the updates that I think these classes need in this blog post.

The first big change is that all of these classes need to more closely adapt to the recent standards in reproducible research. We do some of this already, but need to do more and need to make it more explicit. This means that for every data set that the students use, they need to produce a data dictionary. And as they modify the data sets, they need to produce a changelog file.

Most of the data sets that I use in these classes already have good data dictionaries, but I want the students to cut-and-paste the relevant portions into their own text file. They should then add a bit of supplementary information, if needed. Here’s an example of a data dictionary for the first file that I use in my Introduction to R class. It is scarcely better than the data dictionary found at the second link described in this file, but it’s important for students to see the importance of creating a data dictionary that is stored locally.

~/introduction-to-r-part1/doc/data_dictionary_fd.txt
Written by Steve Simon

See ~/introduction-to-r-part1/README.md for an overview of everything.

This file was downloaded from

http://www.amstat.org/publications/jse/datasets/fat.dat.txt.

A description of the file appears at

https://ww2.amstat.org/publications/jse/datasets/fat.txt.

It represents a study of body fat and body circumference measurements
on 252 men. The data was first used in
Penrose, K., Nelson, A., and Fisher, A. (1985), "Generalized Body 
Composition Prediction Equation for Men Using Simple Measurement 
Techniques" (abstract), Medicine and Science in Sports and Exercise,
17(2), 189.

and later described in a publication

Johnson (1996) "Fitting Percentage of Body Fat to Simple Body
Measurements" Journal of Statistics Education.

and stored in the data archive for this journal. The data set is 
freely available for re-use for non-commercial purposes.

There are 252 rows and 19 columns in this data set. It is stored as a
text file with tab delimiters. There are no missing value codes in this data set.

case, Case Number, a sequential number from 1 to 252.
fat.b, Percent body fat using Brozek's equation, 457/Density - 414.2
fat.s, Percent body fat using Siri's equation, 495/Density - 450
dens, Density (gm/cm^3)
age, Age (yrs)
wt, Weight (lbs)
ht, Height (inches)
bmi, Adiposity index = Weight/Height^2 (kg/m^2)
ffw, Fat Free Weight = (1 - fraction of body fat) * Weight, using Brozek's formula (lbs)
neck, Neck circumference (cm)
chest, Chest circumference (cm)
abdomen, Abdomen circumference (cm) "at the umbilicus and level with the iliac crest"
hip, Hip circumference (cm)
thigh, Thigh circumference (cm)
knee, Knee circumference (cm)
ankle, Ankle circumference (cm)
biceps, Extended biceps circumference (cm)
forearm, Forearm circumference (cm)
wrist, Wrist circumference (cm) "distal to the styloid processes"

There are no hard and fast rules about what goes in a data dictionary, but here are the elements that I included in this particular data dictionary.

• Where the data came from, including urls and references, if available
• A brief description of the data
• Licensing information for this data
• The format of the data, including information about delimiters, if appropriate
• The number of rows and columns in the data
• The codes for missing values

The data dictionary also includes an entry for each variable with the name of the variable, a brief description, and units of measurement, if appropriate. There are no categorical variables in this data set, but if there were, you should include the values used and what they represent (e.g., 1=male, 2=female).

A data dictionary does not have to include all of the things I listed above and it can include things that I did not include. Use your best judgement to decide how much to document in the data dictionary.

The other thing I want to emphasize in these classes is a changelog file. This file provides a historical record when new programs are written and when new data sets are created. Here’s a fictional example of a changelog file.

~/introduction-to-r/doc/changelog.txt
Written by Steve Simon

See ~/introduction-to-r-part1/README.md for an overview of everything.

## 2018-02-20
Created src, doc, data, results directories and moved files into them.
## 2016-08-22
Enhanced documentation of part1a.Rmd.
## 2016-08-18
Created part1a.Rmd through part1d.Rmd by splitting part1.Rmd.
## 2016-08-08
Updated README.md
Export fd to a text file, body.txt
## 2016-05-31
Created github repository.
Created README.md 
Created part1.Rmd
Imported data and stored it as fd in part1.RData
Converted outlier in fd to NA and stored as fd1 in part1.RData
Removed row with outlier from fd and stored as fd2 in part1.RData

Now, I don’t have a changelog.txt file for any of my projects. I rely on git to produce something equivalent to a changelog file, and pulled a small fraction of the changes documented by git into the file above.

Git does not work as well for SAS and SPSS as it does for R, so it makes more sense to teach how to create a changelog file in all three classes.