Projects In Data

The DOHMH New York City Restaurant Inspection dataset provided a close to comprehensive list of all restaurants operating in New York City along with their corresponding addresses and cuisine categories. Two primary statistics were derived using this data, restaurant density and restaurant diversity. Restaurant density is a measure of how many restaurants are located within in a given area, while restaurant diversity is a measure of how many different categories of restaurants are located in a given area. At first restaurants were grouped by ZIP codes to calculate these metrics, however to make the data more digestible the decision was made to first map these different ZIP codes to New York City neighborhoods using a dataset from the NYC Dept. of Health and then group the restaurants by neighborhood to generate the necessary statistics.

There arose a need to normalize the restaurant density metric by population size and geographic area after realizing that there would likely be more restaurants in areas where there are more people and/or land. To compare restaurant densities across different neighborhoods two separate statistics were generated using the 2010 US Population Density dataset, they were restaurant density per 100 people, and restaurant density per square mile. The same was not done for restaurant diversity as it was unclear if a larger population or landmass would increase the number of different categories of restaurants, as it is unclear what can attribute to an area having more categories of restaurants.

All of this data was paired with rent data provided by the Zillow Rent Index which contained information on the monthly median rent for ZIP codes across the US, spanning from November of 2010 to March of 2017. Median rent data from the most recent month (March 2017) was used in this study when talking about the current rent of a specific area. The ZIP codes were grouped by neighborhood in the same manner as the restaurant ZIP codes and the average of rent of all the ZIP codes in a specific neighborhood was taken as the average rent for that neighborhood. A replication of the Zillow Rent Index time series was created with just the ZIP codes from New York grouped by neighborhood in a similar manner as described above.

Following the presentation on the data from rent and diversity/density, Yelp data on prices and ratings was gathered for two case studies. This was collected by scraping Yelp.com using a Python Beautiful Soup script which can be found under downloads. The web scraper searched for restaurants by address and returned their price ranking and ratings when found. The decision to search by addresses was made because of the large amount of variability in how restaurants were named in DOHMH dataset and on the Yelp website; addresses were thought to be more concrete and uniform when translating from the DOHMH dataset to Yelp. However, the web scraper still encountered discrepancies when searching by addresses and therefore was only 60% successful in returning data for the addresses included in the case study. In the end this still left over 800 points of data to be used in the case study analysis.

Firstly, using the population density and overall levels, a density-adjusted moving average was created to analyze the average amount of rent a New Yorker pays. This was compared against a time series of rent levels for both Greenwich Village/Soho and the Northeast Bronx, as is pictured below.

Using descriptive statistics, some general statements are able to be made about these two samples. Greenwich Village had an average rent of 3 standard deviations above the city mean of $2343. Relatedly, Greenwich Village also has the fourth highest diversity. The restaurant density was again 3 standard deviations above the mean.

Perhaps more strikingly though is the Northeast Bronx. Despite being only 1 standard deviation below the mean, the density of restaurants is only 4% that of Greenwich Village and Soho. The Northeast Bronx is also 5.4 times as large, but has 21% the absolute number of restaurants.

The ‘$’ symbol is used to denote the price range of a restaurant. Further definition of Yelp’s symbol is as follows:

$ = Under $10   (“Inexpensive”)

$$ = $11-$30   (“Moderate”)

$$$ = $31-$60   (“Pricey”)

$$$$ = Above $61   (“Ultra High-End”)

A side-by-side examination of the two locations reveals that Greenwich/Soho has a greater number of high priced restaurants compared to the Northeast Bronx. Surprisingly, Northeast Bronx does not have a single restaurant in the top two tiers. However, the most common restaurants in both neighborhoods fall under the inexpensive and moderate price tiers.

Similarly to the price tiers analysis, the ratings breakdown shows that a greater percentage of restaurants in the Greenwich/ Soho neighborhood fall in the upper range of the rating spectrum compared to Northeast Bronx. However, differing from above is the absence of restaurants in Greenwich/ Soho that fall in the lower portion of the ratings tier. A possible explanation for this can be that restaurants with lower ratings do not attract as many customers- and hence their money. Consequently, lower rated restaurants cannot afford to pay rent in Greenwich/ Soho, which we learned earlier to be among the highest in New York.

Realizing the strength of this relationship, we explored other potential correlative factors. When restaurant diversity (the number of cuisines) was measured against rent, a similar relationship was found, again with a very high R²=50.21%. From these two measures we can draw some general statements: high rent neighborhoods are more likely to have greater restaurant density and diversity. Similarly, low rent neighborhoods are more likely to have low restaurant density and diversity. The X-axis marks the median rent level and the Y-axis the diversity.

Lastly, in comparing the relative densities of both restaurants and populations, a positive trend that was found for a majority of the points is linearly correlated. This covariance (at least of the left side) indicated that denser population centers are also denser restaurant centers. The X-axis marks restaurant density and the Y-axis population density.

When this is analyzed alongside a graph comparing population density to rent, the again positive correlation hints to us that this relationship between rent, population density, and restaurant density are all strongly interconnected. The X-axis marks population density and the Y-axis median rent level.