How to Gather Data Using a Web Crawler: An Application Using SAS to Search Edgar

14 Pages Posted: 9 Oct 2007

See all articles by Joseph Engelberg

Joseph Engelberg

University of California, San Diego (UCSD) - Rady School of Management

Srinivasan Sankaraguruswamy

National University of Singapore (NUS) - Department of Accounting

Date Written: 2007

Abstract

Empirical researchers in economics, finance and accounting grapple with a fundamental problem of gathering a representative sample of firms to test various theories that abound in the literature. Several well established data vendors provide a wealth of information toward this endeavor. However, data providers only provide a portion of information available. In this paper we provide a simple SAS program that can search for particular phrases in any Form filed by a registrant with the SEC. This allows researchers to "crawl" the web and access a large trove of data disclosed by managers in their public filings.

Keywords: Data gathering, Web crawling

JEL Classification: G00, M40, M41, M46

Suggested Citation

Engelberg, Joseph and Sankaraguruswamy, Srinivasan, How to Gather Data Using a Web Crawler: An Application Using SAS to Search Edgar (2007). Available at SSRN: https://ssrn.com/abstract=1015021 or http://dx.doi.org/10.2139/ssrn.1015021

Joseph Engelberg

University of California, San Diego (UCSD) - Rady School of Management ( email )

9500 Gilman Drive
Rady School of Management
La Jolla, CA 92093
United States

Srinivasan Sankaraguruswamy (Contact Author)

National University of Singapore (NUS) - Department of Accounting ( email )

1 Business Link
Singapore, 117592
Singapore
+65 6874 4473 (Phone)
+65 6779 2083 (Fax)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
4,248
Abstract Views
15,358
Rank
4,440
PlumX Metrics