SAS has some very powerful ways to find duplicate values and to store the duplicates separate from the unique values. Many of these use the sort procedure. Here is a nice guideline for what would otherwise be very difficult to figure out on your own. Continue reading
Recommended: Bibliographies and citations (in R Markdown)
If you want to use R Markdown to prepare research papers and presentations, you need to learn how to cite references and include a bibliography. This is a nice introduction and shows the variety of options at your fingertips. Continue reading
Recommended: What would Florence Nightingale make of big data?
This is a nice video, professionally produced and very short (4 minutes) that shows the importance Florence Nightingale attached to Statistics. It reviews how she used Statistics aggressively to lobby for improvements to health care, and speculates on what she would think about the efforts today to use big data for decision making. The narrator is David Spiegelhalter, a famous statistician. Continue reading
Recommended: Writing a discussion section: how to integrate substantive and statistical expertise
The paper in BMC Medical Research Methodology gives practical step by step guidance on writing your discussion section. Continue reading
Recommended: When Big Data goes to school
File this under the “dark side” of data science. Alfie Kohn is a critic of many of the motivational methods used in business and education, and he makes many good points in this blog post about relying on readily available data without questioning its quality. Continue reading
Recommended: Copyright and Data
This is a nice one page document on copyright issues associated with research data and how to best share your research data with others. Continue reading
Recommended: Case for omitting tied observations in the two-sample t-test and the Wilcoxon-Mann-Whitney Test
When you are running a non-parametric test, like the Wilcoxon-Mann-Whitney test, you can only be 100% of the properties of that test (including Type I and Type II error rates) if the data are continuous. If there are ties in the data, the properties of the test are unknown. This paper shows four commonly used approaches for settings where values might be tied and runs simulations to measure Type I and Type II error rates for both the two-sample t-test and the Wilcoxon-Mann-Whitney test under a range of tied values and a range of distributions. The results are, at least to me, quite surprising. Continue reading
Recommended: When life gives you coloured cells, make categories
A lot of people use formatting to denote important information in an Excel spreadsheet. In particular, they will use the color of a cell to designate a particular category. Pretty much all formatting information is lost when you import from Excel to R or any other statistical package. But rather than ask people to go back and fix things, there are a few tricks that you can use to recover this information, as is shown in this blog entry. Continue reading
Recommended: Ethical Practice in Data Mining
This is a very nice summary of six major areas where data mining has led to serious ethical concerns. Continue reading
PMean: Writing the introduction section of a research thesis or dissertation
The introduction section of your research thesis or dissertation is the first thing that most people will read after reading the abstract. Some people use the introduction section to provide a literature review, and I won’t talk about that here. I did offer a nice recommendation on how to write a literature review in an earlier post. The introduction should provide present your research problem (research question, research hypothesis), but first you have to offer some context. Continue reading