Hướng dẫn when should you bootstrap statistics? - khi nào bạn nên bootstrap thống kê?

Comparing the bootstrapping approach to the traditional approach, and understanding why it’s useful.

Statistics is the science of learning from data. Statistical knowledge aids in the proper methods for collecting data, using correct methods for analyzing data, and effectively presenting the results derived from data. These methods are crucial to making decisions and predictions, whether it be predicting the consumer demand for a product; using text-mining to filter spam emails; or making real-time decisions in self-driving cars. Most times when conducting research, it is impractical to collect data from the population. This can be because of budget and/or time constraints, among other factors. Instead, a subset of the population is taken, and insight is gathered from that subset to learn more about the population. It means then that suitably accurate information can be obtained quickly and relatively inexpensively from an appropriately drawn sample. However, many things can affect how well a sample reflects the population; and therefore, how valid and reliable the conclusions will be. Because of this, let us talk about bootstrapping statistics.

“Bootstrapping is a statistical procedure that resamples a single dataset to create many simulated samples. This process allows for the calculation of standard errors, confidence intervals, and hypothesis testing” (Forst). A bootstrapping approach is an extremely useful alternative to the traditional method of hypothesis testing as it is fairly simple and it mitigates some of the pitfalls encountered within the traditional approach, which will be discussed later. Statistical inference generally relies on the sampling distribution and the standard error of the feature of interest. The traditional approach (or large sample approach) draws one sample of size n from the population, and that sample is used to calculate population estimates to then make inferences on. Now, in reality, only one sample has been observed. However, there is the idea of a sampling distribution, which is a theoretical set of all possible estimates if the population were to be resampled. The theory states that, under certain conditions such as large sample sizes, the sampling distribution will be approximately normal, and the standard deviation of the distribution will be equal to the standard error. But what happens if the sample size is not sufficiently large? Then, it cannot necessarily be assumed that the theoretical sampling distribution is normal. This then makes it difficult to determine the standard error of the estimate, and harder to draw reasonable conclusions from the data.

As with the traditional approach, a sample of size n is drawn from the population within the bootstrapping approach. Let us call this sample S. Then, rather than using theory to determine all possible estimates, the sampling distribution is created by resampling observations with replacement from S, m times, with each resampled set having n observations. Now, if sampled appropriately, S should be representative of the population. Therefore, by resampling S m times with replacement, it would be as if m samples were drawn from the original population, and the estimates derived would be representative of the theoretical distribution under the traditional approach. It must be noted that increasing the number of resamples, m, will not increase the amount of information in the data. That is, resampling the original set 100,000 times is not more useful than only resampling it 1,000 times. The amount of information within the set is dependent on the sample size, n, which will remain constant throughout each resample. The benefit of more resamples, then, is to derive a better estimate of the sampling distribution.

Now that we understand the bootstrapping approach, it must be noted that the results derived are basically identical to those of the traditional approach. Additionally, the bootstrapping approach will always work because it does not assume any underlying distribution of the data. This contrasts with the traditional approach which theoretically assumes that the data are normally distributed. Knowing how the bootstrapping approach works, a logical question to arise is “does the bootstrapping approach rely too much on the observed data?” This is a good question, given that the resamples are derived from the initial sample. And because of this, it is logical to assume that an outlier will skew the estimates from the resamples. Although this is true, if the traditional approach is considered, it will be seen that an outlier within the dataset will also skew the mean and inflate the standard error of the estimate. Therefore, while it might be tempting to think that an outlier can show up multiple times within the resampled data and skew the results, thus making the traditional approach better, the bootstrapping approach relies as much on the data as does the traditional approach. “The advantages of bootstrapping are that it is a straightforward way to derive the estimates of standard errors and confidence intervals, and it is convenient since it avoids the cost of repeating the experiment to get other groups of sampled data. Although it is impossible to know the true confidence interval for most problems, bootstrapping is asymptotically consistent and more accurate than using the standard intervals obtained using sample variance and the assumption of normality” (Cline).

Cả hai cách tiếp cận đều yêu cầu sử dụng các mẫu được vẽ phù hợp để suy luận về quần thể. Tuy nhiên, sự khác biệt lớn nhất giữa hai phương pháp này là cơ chế đằng sau ước tính phân phối lấy mẫu. Quy trình truyền thống yêu cầu người ta phải có một thống kê kiểm tra đáp ứng các giả định cụ thể để đạt được kết quả hợp lệ và điều này chủ yếu phụ thuộc vào thiết kế thử nghiệm. Cách tiếp cận truyền thống cũng sử dụng lý thuyết để cho biết phân phối lấy mẫu sẽ trông như thế nào, nhưng kết quả sụp đổ nếu các giả định của lý thuyết không được đáp ứng. Phương pháp bootstrapping, mặt khác, lấy dữ liệu mẫu ban đầu và sau đó lấy lại nó để tạo ra nhiều mẫu [mô phỏng]. Cách tiếp cận này không dựa vào lý thuyết vì phân phối lấy mẫu có thể được quan sát đơn giản, và người ta không phải lo lắng về bất kỳ giả định nào. Kỹ thuật này cho phép ước tính chính xác số liệu thống kê, điều này rất quan trọng khi sử dụng dữ liệu để đưa ra quyết định.

Citations:

Cline, Graysen. Phương pháp thống kê phi tham số sử dụng R. United Kingdom, Edtech, 2019.

Forst, Jim. Giới thiệu về bootstrapping trong số liệu thống kê với một ví dụ. Thống kê của Jim. https://statisticbyjim.com/hypothesis-testing/bootstrapping/. Ngày truy cập: ngày 17 tháng 6 năm 2020.

References:

Brownlee, Jason. Một phần giới thiệu nhẹ nhàng về phương pháp bootstrap. Machine Learning Mastery, ngày 25 tháng 5 năm 2018. https://machinelearningmastery.com/a-gentle-introduction-to-the-bootstrap-method/. Ngày truy cập: ngày 24 tháng 5 năm 2020.

Kulesa, Anthony et al. Phân phối lấy mẫu và bootstrap. Phương pháp tự nhiên Vol. 12,6 (2015): 477 bóng8. doi: 10.1038/nmeth.3414

Vật liệu hữu ích khác:

http://faculty.washington.edu/yenchic/17Sp_403/Lec5-

bootstrap.pdfhttps://web.as.uky.edu/statistics/users/pbreheny/764-F11/notes/12-6.pdf

http://www.stat.rutgers.edu/home/mxie/rcpapers/bootstrap.pdf

Khi nào tôi nên sử dụng số liệu thống kê bootstrap?

Tại sao chúng ta lại bootstrap một mô hình thống kê?

Khi nào bạn sẽ sử dụng bài kiểm tra giả thuyết bootstrap?

Khi nào bạn không nên sử dụng bootstrapping?