At the Joint Statistics Meetings, I found out that the advantages of some of the new libraries for data manipulation (like dplyr and tidyr) go beyond just the flexibility of the new methods of data manipulation. These libraries produce code that is easier to read and which also runs a lot faster. I did not appreciate how much faster until I tried a test today. Continue reading
Tag Archives: Data management
Recommended: Organizing data in spreadsheets
I have some guidance on how to organize data (written back in 1999), but these guidelines are far superior. To be honest, you should use a database for anything more than half complicated, but for those simple data sets where you can use a text file or a spreadsheet, Dr. Broman’s comments are very helpful. Continue reading
Recommended: Handling date-times in R
Dates in R, like dates in any other software package, are tricky to work with. Here’s a nice guide that will help you get started. Continue reading
PMean: What should go into a data codebook
Before you start your data entry, you should create a data codebook. If you don’t have a data codebook when you hand your data over to someone else, take the time to create one for their benefit and yours. The data codebook contains a description of your data set. There’s no standard form for a data codebook, and what you describe may depend on a variety of factors, such as the complexity of your data set, the number of people involved in data collection and data entry, and the number of people that you are likely to share your data with. Here are some of the elements that you should think about putting in a data codebook. Continue reading