This was a webscraping project where we had to create a website that would scrape information about mars from various sites at the push of a button.
When a new scrape is requested, the website will find the latest tweet, current weather, current featured image, and show a table of Mars Facts and pictures of Mar’s 4 hemispheres.
We used mongoDB with Flask templating to create the HTML page to display all of the information that was scraped.
This part of the project we were required to scrape using Jupyter Notebook, BeautifulSoup, Pandas, and Requests/Splinter.
Visit the url for JPL’s Featured Space Image here.
Use splinter to navigate the site and find the image url for the current Featured Mars Image
mars_weather.Visit the Mars Facts webpage here and use Pandas to scrape the table containing facts about the planet including Diameter, Mass, etc.
Use Pandas to convert the data to a HTML table string.
Visit the USGS Astrogeology site here to obtain high resolution images for each of Mar’s hemispheres.
Save both the image url string for the full resolution hemipshere image, and the Hemisphere title containing the hemisphere name.
Use a Python dictionary to store the data using the keys img_url and title.
Append the dictionary with the image url string and the hemisphere title to a list. This list will contain one dictionary for each hemisphere.
Use MongoDB with Flask templating to create a new HTML page that displays all of the information that was scraped from the URLs above.
Start by converting your Jupyter notebook into a Python script called scrape_mars.py with a function called scrape that will execute all of your scraping code from above and return one Python dictionary containing all of the scraped data.
Next, create a route called /scrape that will import your scrape_mars.py script and call your scrape function.
Store the return value in Mongo as a Python dictionary.
Create a root route / that will query your Mongo database and pass the mars data into an HTML template to display the data.
Create a template HTML file called index.html that will take the mars data dictionary and display all of the data in the appropriate HTML elements. Use the following as a guide for what the final product should look like, but feel free to create your own design.