With each iteration, your loop is creating new .
Also, if you want to remove space between paragraphs, use p { margin: 0, !important; }
.
You could try this function which gives the HTML
output you wanted.
def printTohtml[Alist, htmlfile]:
html = "\n\np { margin: 0 !important; }\n\n"
title = "Study - User - zip file - Last date modified"
html += '\n' + title + '
\n'
for line in Alist:
para = '' + ', '.join[line] + '
\n'
html += para
with open[htmlfile, 'w'] as f:
f.write[html + "\n\n"]
Alist = [['123', 'user1', 'New Compressed [zipped] Folder.zip', '05-24-17'],
['123', 'user2', 'Iam.zip', '05-19-17'], ['abcd', 'Letsee.zip', '05-22-17'],
['Here', 'whichTwo.zip', '06-01-17']]
printTohtml[Alist, 'zip_files.html']
The markup written to zip_files.html
is:
p { margin: 0 !important; }
Study - User - zip file - Last date modified
123, user1, New Compressed [zipped] Folder.zip, 05-24-17
123, user2, Iam.zip, 05-19-17
abcd, Letsee.zip, 05-22-17
Here, whichTwo.zip, 06-01-17
The page displays the following output:
Study - User - zip file - Last date modified
123, user1, New Compressed [zipped] Folder.zip, 05-24-17
123, user2, Iam.zip, 05-19-17
abcd, Letsee.zip, 05-22-17
Here, whichTwo.zip, 06-01-17
View Discussion
Improve Article
Save Article
View Discussion
Improve Article
Save Article
In this article, we are going to write the output to an HTML file with Python BeautifulSoup. BeautifulSoup is a python library majorly used for web scraping but in this article, we will discuss how to write the output to an HTML file.
Modules needed and installation:
pip install bs4
Approach:
- We will first import all the required libraries.
- Make a get request to the desired URL and extract its page content.
- Using the file data type of python write the output in a new file.
Steps to be followed:
Step 1: Import the required libraries.
Python3
from
bs4
import
BeautifulSoup
import
requests
Step 2: We will perform a get request to the Google search engine home page and extract its page content and make a soup object out of it by passing it to beautiful soup, and we will set the markup as html.parser.
Note: if you are extracting a xml page set the markup as xml.parser
Python3
Step 3: We use the file data type of python and write the soup object in the output file. We will set the encoding to UTF-8. We will use .prettify[] function on soup object that will make it easier to read. We will convert the soup object to a string before writing it.
We will store the output file in the same directory with the name output.html
Python3
with
open
[
"output.html"
,
"w"
, encoding
=
'utf-8'
] as
file
:
file
.write[
str
[soup.prettify[]]]
Below is the full implementation:
Python3
Output:
Contents
- Lesson Goals
- Files Needed For This Lesson
- Building an HTML wrapper
- Metadata
- Python string formatting
- Self-documenting data file
- Mac Instructions
- Windows Instructions
- Putting it all together
- Suggested Readings
- Code Syncing
Lesson Goals
This lesson takes the frequency pairs created in Counting Frequencies and outputs them to an HTML file.
Here you will learn how to output data as an HTML file using Python. You will also learn about string formatting. The final result is an HTML file that shows the keywords found in the original source in order of descending frequency, along with the number of times that each keyword appears.
Files Needed For This Lesson
obo.py
If you do not have these files from the previous lesson, you can download programming-historian-6, a zip file from the previous lesson
Building an HTML wrapper
In the previous lesson, you learned how to embed the message “Hello World!” in HTML tags, write the result to a file and open it automatically in the browser. A program that puts formatting codes around something so that it can be used by another program is sometimes called a wrapper. What we’re going to do now is develop an HTML wrapper for the output of our code that computes word frequencies. We’re also going to add some helpful, dynamic metadata to supplement the frequency data collected in Counting Frequencies.
The distinction between data and metadata is crucial to information science. Metadata are data about data. This concept should already be very familiar to you, even if you haven’t heard the term before. Consider a traditional book. If we take the text of the book to be the data, there are a number of other characteristics which are associated with that text, but which may or may not be explicitly printed in the book. The title of the work, the author, the publisher, and the place and date of publication are metadata that are typically printed in the work. The place and date of writing, the name of the copy editor, Library of Congress cataloging data, and the name of the font used to typeset the book are sometimes printed in it. The person who purchased a particular copy may or may not write their name in the book. If the book belongs in the collection of a library, that library will keep additional metadata, only some of which will be physically attached to the book. The record of borrowing, for example, is usually kept in some kind of database and linked to the book by a unique identifier. Libraries, archives and museums all have elaborate systems to generate and keep track of metadata.
When you’re working with digital data, it is a good idea to incorporate metadata into your own files whenever possible. We will now develop a few basic strategies for making our data files self-documenting. In our wrapper, we want to include dynamic information about the file, such as the time and date it was created, as well as an HTML title that is relevant to the file. In this case we could just give it a name ourselves, but when we start working with multiple files, automatically creating self-documenting files will save a lot of time, so we’ll practice now. And for that, we’ll have to learn to take advantage of a few more powerful string formatting options.
Python string formatting
Python includes a special formatting operator that allows you to insert one string into another one. It is represented by a percent sign followed by an “s”. Open a Python shell and try the following examples.
frame = 'This fruit is a %s'
print[frame]
-> This fruit is a %s
print[frame % 'banana']
-> This fruit is a banana
print[frame % 'pear']
-> This fruit is a pear
There is also a form which allows you to interpolate a list of strings into another one.
frame2 = 'These are %s, those are %s'
print[frame2]
-> These are %s, those are %s
print[frame2 % ['bananas', 'pears']]
-> These are bananas, those are pears
In these examples, a %s
in one string indicates that another string is going to be embedded at that point. There are a range of other string formatting codes, most of which allow you to embed numbers in strings in various formats, like %i
for integer [eg. 1, 2, 3], %f
for floating-point decimal [eg. 3.023, 4.59, 1.0], and so on.
Using this method we can input information that is unique to the file.
Self-documenting data file
Let’s bundle some of the code that we’ve already written into functions. One of these will take a URL and return a string of lowercase text from the web page. Copy this code into the obo.py
module.
# Given a URL, return string of lowercase text from page.
def webPageToText[url]:
import urllib.request, urllib.error, urllib.parse
response = urllib.request.urlopen[url]
html = response.read[].decode['UTF-8']
text = stripTags[html].lower[]
return text
We’re also going to want a function that takes a string of any sort and makes it the body of an HTML file which is opened
automatically in Firefox. This function should include some basic metadata, like the time and date that it was created and the name of the program that created it. Study the following code carefully, then copy it into the obo.py
module.
Mac Instructions
If you are using a Mac, make sure you include the proper file path in the filename variable on the 2nd last line to reflect where you’re saving your files.
# Given name of calling program, a url and a string to wrap,
# output string in html body with basic metadata and open in Firefox tab.
def wrapStringInHTMLMac[program, url, body]:
import datetime
from webbrowser import open_new_tab
now = datetime.datetime.today[].strftime["%Y%m%d-%H%M%S"]
filename = program + '.html'
f = open[filename,'w']
wrapper = """
%s output - %s
URL: %s
%s
"""
whole = wrapper % [program, now, url, url, body]
f.write[whole]
f.close[]
#Change the filepath variable below to match the location of your directory
filename = 'file:///Users/username/Desktop/programming-historian/' + filename
open_new_tab[filename]
Windows Instructions
# Given name of calling program, a url and a string to wrap,
# output string in html body with basic metadata
# and open in Firefox tab.
def wrapStringInHTMLWindows[program, url, body]:
import datetime
from webbrowser import open_new_tab
now = datetime.datetime.today[].strftime["%Y%m%d-%H%M%S"]
filename = program + '.html'
f = open[filename,'w']
wrapper = """
%s output - %s
URL: %s
%s
"""
whole = wrapper % [program, now, url, url, body]
f.write[whole]
f.close[]
open_new_tab[filename]
***
Note that this function makes use of the string formatting operator about which we just learned. If you are still having trouble with this idea, take a look at the HTML file that opened in your new Firefox tab and you should see how this worked. If you’re still stuck, take a look at the
URL: //www.oldbaileyonline.org/browse.jsp?id=t17800628-33&div=t17800628-33
in the HTML file and trace back how the program knew to put the URL value there.
The function also calls the Python datetime
library
to determine the current time and date. Like the string formatting operator %s
, this library uses the %
as replacements for values. In this case, the %Y %m
%d %H %M %S
represents year, month, date, hour, minute and second respectively. Unlike the %s
, the program will determine the value of these variables for you using your computer’s clock. It is important to recognize this difference.
This date metadata, along with the name of the program that called the function, is stored in
the HTML title tag. The HTML file that is created has the same name as the Python program that creates it, but with a .html
extension rather than a .py
one.
Putting it all together
Now we can create another version of our program to compute frequencies. Instead of sending its output to a text file or an output window, it sends the output to an HTML file which is opened in a new Firefox tab. From there, the program’s output can be added easily as
bibliographic entries to Zotero. Type or copy the following code into your text editor, save it as html-to-freq-3.py
and execute it, to confirm that it works as expected.
Use either obo.wrapStringInHTMLMac[] or obo.wrapStringInHTMLWindows[] as appropriate for your system.
# html-to-freq-3.py
import obo
# create sorted dictionary of word-frequency pairs
url = '//www.oldbaileyonline.org/browse.jsp?id=t17800628-33&div=t17800628-33'
text = obo.webPageToText[url]
fullwordlist = obo.stripNonAlphaNum[text]
wordlist = obo.removeStopwords[fullwordlist, obo.stopwords]
dictionary = obo.wordListToFreqDict[wordlist]
sorteddict = obo.sortFreqDict[dictionary]
# compile dictionary into string and wrap with HTML
outstring = ""
for s in sorteddict:
outstring += str[s]
outstring += "
"
obo.wrapStringInHTMLMac["html-to-freq-3", url, outstring]
Note that we interspersed our word-frequency pairs with the HTML break tag , which acts as a newline. If all went well, you should see the same word frequencies that you computed in the
last section, this time in your browser window.
Suggested Readings
- Lutz, Learning Python
- Re-read and review Chs. 1-17
Code Syncing
To follow along with future lessons it is important that you have the right files and programs in your “programming-historian” directory. At the end of each chapter you can download the “programming-historian” zip file to make sure you have the correct code. If you are
following along with the Mac / Linux version you may have to open the obo.py
file and change “file:///Users/username/Desktop/programming-historian/” to the path to the directory on your own computer.
- python-lessons7.zip zip sync