ANDROID JSOUP
Android Jsoup: A Complete Guide to HTML Parsing in Android
Jsoup is a powerful Java library used for working with real-world HTML. It provides an easy-to-use API for extracting and manipulating data, and it’s commonly used in Android apps to parse HTML documents, scrape web pages, and extract useful information.
In this guide, we will explore how to use Jsoup in Android to parse HTML, extract data, and manipulate content. Let’s dive into it!
What is Jsoup?
Jsoup is a Java library that makes it easy to scrape and parse HTML from the web. You can use it to:
- Parse HTML documents from a string, file, or URL.
- Extract and manipulate the elements from HTML.
- Clean and sanitize HTML.
Jsoup provides methods to:
- Select elements with CSS selectors.
- Traverse and manipulate the DOM (Document Object Model).
- Get or modify HTML or text inside elements.
Step 1: Adding Jsoup to Your Android Project
To use Jsoup in your Android project, you need to add it as a dependency in your build.gradle file. Open your app/build.gradle file and add the following line to the dependencies section:
dependencies {
implementation 'org.jsoup:jsoup:1.15.3'
}
Then sync the Gradle files to download the Jsoup library.
Step 2: Basic Usage of Jsoup in Android
Now that we’ve added Jsoup to our project, let’s start by parsing a simple HTML document.
Sample HTML
Let’s assume we have the following HTML content:
<html>
<body>
<h1>Welcome to Jsoup</h1>
<p>This is a simple HTML page to demonstrate Jsoup parsing.</p>
<a href="https://www.example.com">Visit Example</a>
</body>
</html>
You can parse this HTML and extract the data using Jsoup.
Step 3: Parse HTML from a String
To parse the HTML from a string, you can use the Jsoup.parse() method.
Here’s an example of how to parse this HTML content:
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
public class JsoupExample {
public void parseHtmlString() {
String htmlContent = "<html><body><h1>Welcome to Jsoup</h1><p>This is a simple HTML page to demonstrate Jsoup parsing.</p><a href='https://www.example.com'>Visit Example</a></body></html>";
// Parse the HTML string
Document doc = Jsoup.parse(htmlContent);
// Extract elements by CSS selectors
String title = doc.select("h1").text(); // Extracts text inside <h1> tag
String paragraph = doc.select("p").text(); // Extracts text inside <p> tag
String link = doc.select("a").attr("href"); // Extracts the href attribute of <a> tag
// Print the results
System.out.println("Title: " + title);
System.out.println("Paragraph: " + paragraph);
System.out.println("Link: " + link);
}
}
Explanation:
Jsoup.parse(): This method parses the HTML content and returns aDocumentobject.doc.select(): This method allows you to use CSS selectors to extract specific elements.text(): This method extracts the text content from an element.attr(): This method retrieves the value of a specified attribute (likehreffor links).
Step 4: Parsing HTML from a URL
Jsoup can also parse HTML directly from a URL. For this, you will need to perform network operations. Always remember to run network operations in a background thread (e.g., using AsyncTask, ExecutorService, or Kotlin Coroutines) to avoid blocking the UI thread.
Here’s how to fetch and parse HTML from a URL using Jsoup:
Example of Parsing HTML from a URL:
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import java.io.IOException;
public class JsoupUrlExample {
public void parseHtmlFromUrl() {
String url = "https://www.example.com"; // Replace with the URL you want to scrape
// Make an HTTP request and parse the HTML
new Thread(() -> {
try {
// Connect to the URL and parse the document
Document doc = Jsoup.connect(url).get();
// Extract the title of the page
String title = doc.title(); // Extracts the <title> tag's content
// Print the title
System.out.println("Page Title: " + title);
} catch (IOException e) {
e.printStackTrace();
}
}).start();
}
}
Explanation:
Jsoup.connect(url).get(): This method connects to the specified URL and retrieves the HTML content. It returns aDocumentobject, which you can then parse and manipulate.
Note: Always handle network requests in the background to avoid blocking the UI thread.
Step 5: Manipulating HTML with Jsoup
Jsoup makes it easy to manipulate HTML content. You can change text, add or remove elements, and modify attributes.
Example of Modifying HTML:
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
public class JsoupManipulateExample {
public void manipulateHtml() {
String htmlContent = "<html><body><h1>Old Title</h1><p>Old paragraph content.</p></body></html>";
// Parse the HTML
Document doc = Jsoup.parse(htmlContent);
// Change the text inside <h1> tag
doc.select("h1").text("New Title");
// Change the text inside <p> tag
doc.select("p").text("New paragraph content.");
// Add a new paragraph
doc.body().append("<p>This is a new paragraph added by Jsoup.</p>");
// Print the modified HTML
System.out.println(doc.html());
}
}
Explanation:
select(): Use CSS selectors to find elements.text(): Set new text content for the selected element.append(): Add new content to the selected element (in this case, the body).
Step 6: Extracting Data from Tables with Jsoup
A common use case for Jsoup is extracting data from HTML tables. You can easily scrape tables and get their data.
Example: Extracting Data from a Table
Consider the following HTML table:
<table>
<tr><th>Name</th><th>Age</th></tr>
<tr><td>John Doe</td><td>30</td></tr>
<tr><td>Jane Smith</td><td>25</td></tr>
</table>
To extract data from this table:
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class JsoupTableExample {
public void extractDataFromTable() {
String htmlContent = "<table><tr><th>Name</th><th>Age</th></tr><tr><td>John Doe</td><td>30</td></tr><tr><td>Jane Smith</td><td>25</td></tr></table>";
// Parse the HTML content
Document doc = Jsoup.parse(htmlContent);
// Select all rows in the table
Elements rows = doc.select("table tr");
// Loop through the rows and extract the data
for (Element row : rows) {
// Select each column (td) in the row
Elements cols = row.select("td");
if (!cols.isEmpty()) {
String name = cols.get(0).text();
String age = cols.get(1).text();
// Print the extracted data
System.out.println("Name: " + name + ", Age: " + age);
}
}
}
}
Explanation:
select("table tr"): Selects all rows (<tr>) inside the table.select("td"): Selects the data cells (<td>) in each row.text(): Extracts the text content inside the<td>cells.
Step 7: Handling JavaScript with Jsoup
Jsoup can only parse HTML, not JavaScript. If your content depends on JavaScript (e.g., dynamic content loaded via JavaScript), you may need to use other libraries like Selenium or Puppeteer for scraping such pages.
Step 8: Conclusion
Jsoup is a powerful and efficient library for parsing and manipulating HTML content in Android. Whether you’re scraping data from web pages or cleaning up HTML, Jsoup offers a simple and intuitive API for handling HTML documents. By following the steps and examples in this guide, you should now be able to parse and manipulate HTML content with ease in your Android apps.
Remember to always perform network requests on a background thread to avoid blocking the UI, and use appropriate error handling to manage network-related issues.
0 Comments