5. Progress Report
The current focus of Illusory Wall is to get the user functionality up and working. This also includes the ability to restrict decisive endpoints to admin uses as they should be the only ones to modify the data. This requires a slight reworking of the current endpoints, but nothing to be considered severe. This mostly just means creating a system of generating authorization tokens on successful logins that allow the logged in user to use certain functionalities of the application. Along with this the current database schema had to be adjusted to include a "Users" table. This encompasses the the unique username, user ID, the encrypted password, and the salt for the password encryption.
One of our goals for basic functionality was not only allowing the user to search for an enemy using a substring of its name, but also be able to search for them using "advanced filters." These filters are by classification, whether or not it respawns, HP, and souls dropped in addition to searching by name. This allows for more specific searches and particular queries that can be made to find enemies.
To allow the user to use this advanced search functionality, next to the normal enemy name search bar is an "Advanced" button, which, when clicking, reveals the additional search parameters the user can interact with. As long as the advanced search is visible when the user makes the request to the API, the additional search information will be sent.
Example of an advanced search query
The current advanced functionality being worked on is the web scraping and the generated hitlists. The problem is that the hitlists rely on the basic implementation of the account functionality so that the hitlists can be attached to their users. Besides that slight road block the development for the hitlists is under way while the development for the web scraper is wrapping up. It recently has been mounted to the client-side UI, giving it a nice interface to interact with. It also allows the admins to fill in any data that is missing or double check the data that actually was gathered.
For the majority of the project, most of the API endpoints were done through a single Enemy controller, since all of our endpoints thus far have only been concerned with accessing, modifying, creating, and deleting records in the Enemy table of our database. Implementing the web scraper, since it operates as a separate entity compared to the Enemy controller, required the introduction of a new WebScraper controller. This controller had to responsibility of a single endpoint: GET /WebScraper/?url=<url>, where a URL to a "darksouls.fandom.com" page is provided as the only parameter in the endpoint. The controller then invokes the separate web scraper library that we have developed, that has the job of validating the passed URL, fetching the page at that URL, and then scraping the data before binding it to an Enemy model and returning it back in the request response.
With the API endpoint and web scraper library created, we used the endpoint to connect the client UI with with API. On the "Add Enemy" page of the client, there is a new button with the text "Scrape from Fandom.com". After clicking on it, the user is prompted with a modal containing a form with one text input, asking for the URL to the enemy page on "darkouls.fandom.com". After the user provides the URL and submits the form, the request is made to the server to scrape the content of that page, which is returned to the client. The returned data is then used to prefill the form that user would normally have to manually fill, allowing for faster population of data in the database.
Empty "Add Enemy" page with new web scraper button.
New web scraper modal where the user will input the URL to a darksouls.fandom.com enemy page.
"Add Enemy" form after the data has been scraped and prefilled.
After completing work on the user account functionality, some advanced features such as the generated hunting list of enemies for the game can be worked on, because that requires it to be attached to an account. Also since the web scraping feature is now mounted to the client side this will allow the database to be filled with data easier and quicker than before, allowing us to produce more real data than what was presented during the demo presentation. With more real data incorporated into the database the work on developing the chart and graph advanced functionality can continue with plenty of testing data to work on.