Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't drop Swagger tables when scraping #1070

Open
iuliaturc opened this issue Jan 17, 2025 · 1 comment
Open

Don't drop Swagger tables when scraping #1070

iuliaturc opened this issue Jan 17, 2025 · 1 comment
Assignees

Comments

@iuliaturc
Copy link

Problem Description
Firecrawl is very useful for scraping API documentation pages. However, it seems to drop Swagger tables, which contain the bulk of the information (endpoints, parameters, etc.)

Proposed Feature
Keep the content of Swagger tables in the scrape.

Use Case
Sample documentation page for which the Swagger table is dropped: https://aviation-edge.com/developers/#delaya

Firecrawl scrape:

[Skip to content](https://aviation-edge.com/developers/#content)

Getting Started

[Introduction](https://aviation-edge.com/developers/#intro)

Dynamic Data

[Flight Tracker API](https://aviation-edge.com/developers/#flight_tracker) [Flight Schedules API](https://aviation-edge.com/developers/#schedulesa) [Historical Schedules API](https://aviation-edge.com/developers/#historicala) [Future Schedules API](https://aviation-edge.com/developers/#futurea) [Airline Routes API](https://aviation-edge.com/developers/#routesa) [Flight Delay API](https://aviation-edge.com/developers/#delaya) [Nearby API](https://aviation-edge.com/developers/#nearbya) [Autocomplete API](https://aviation-edge.com/developers/#autocompletea) [Satellite Tracker API](https://aviation-edge.com/developers/#satellitea)

Static Data

[Airports](https://aviation-edge.com/developers/#airportsa)

[Airlines](https://aviation-edge.com/developers/#airlinesa)

[Airc](https://aviation-edge.com/developers/#airplanesa) [raft](https://aviation-edge.com/developers/#airplanesa)

[Cities](https://aviation-edge.com/developers/#citiesa)

[Countries](https://aviation-edge.com/developers/#countriesa)

[Taxes](https://aviation-edge.com/developers/#taxesa)

Getting Started

[Introduction](https://aviation-edge.com/developers/#intro)

Dynamic Data

[Flight Tracker API](https://aviation-edge.com/developers/#flight_tracker)

[Flight Schedules AP](https://aviation-edge.com/developers/#schedulesa) [I](https://aviation-edge.com/developers/#schedules)

[Airline Routes API](https://aviation-edge.com/developers/#airline_routes)

[Flight Delay API](https://aviation-edge.com/developers/#delay)

[Historical Schedules API](https://aviation-edge.com/developers/#historicala)

[Nearby API](https://aviation-edge.com/developers/#nearbya)

[Autocomplete API](https://aviation-edge.com/developers/#autocompletea)

[Future Schedules API](https://aviation-edge.com/developers/#future)

[Satellite Tracker API](https://aviation-edge.com/developers/#satellitea)

Static Data

[Airports](https://aviation-edge.com/developers/#airportsa)

[Airlines](https://aviation-edge.com/developers/#airlinesa)

[Airc](https://aviation-edge.com/developers/#airplanesa) raft

[Cities](https://aviation-edge.com/developers/#citiesa)

[Countries](https://aviation-edge.com/developers/#countriesa)

[Taxes](https://aviation-edge.com/developers/#taxesa)

# Aviation Edge API for Developers

Our database is divided between static and dynamic data. Static data is that what does not frequently change but due to the massive size of our database, there are constant updates. Think about airport and airline information. The dynamic data is that what changes more frequently or uses our software to turn it into useful tools. Think about flight tracking and airport timetables.

It’s a REST API so depending on the development language you use, you can find the suitable method to develop it. A way to try the API is by getting your [free API key](https://aviation-edge.com/free-api-key/) and then testing it on our [testing page](https://aviation-edge.com/get.php).

[Get Free API Key](https://aviation-edge.com/premium-api/)

[Test Your API Key](https://aviation-edge.com/get.php)

You can find the examples of each endpoint below. The layout of the GET requests are as follows:

**For an entire dataset, when available:**

```javascript
https://aviation-edge.com/v2/public/[ENDPOINT]?key=[API KEY]

Copy

For specific string value information:

https://aviation-edge.com/v2/public/[ENDPOINT]?key=[API KEY]&[STRING]=[VALUE]

Copy

Dynamic Data

Flight Tracker API

Get all global flight information in 1 API call or track individual flights. You can also call all flights based on a single Airline or base it on the departure or arrival Airport.

Servers

https://aviation-edge.com/v2/public/

Authorize

Flight Tracking API

GET/flights

Schemas

FlightTrackingResponse

Schedules API

Get airport arrival and departure schedules of flights information including delay and cancellations. Updated constantly.

Servers

https://aviation-edge.com/v2/public/

Authorize

Flight Schedules API

GET/timetable

Schemas

ScheduleResponse

Translations

Using the translations function in the API with &lang= will slow down the API calls when it involves a lot of translations of airports, cities and countries. Therefor it is recommended to use it in combination with other filters to reduce the amount of data you receive in the response.

Available languages

Code Language
fr French
ar Arabic
fa Persian
ru Russian
zh_hans Chinese Simplified
zh_hant Chinese Traditional
es Spanish
hi Hindi
pt Portuguese
ja Japanese
de German
jv Javanese
ko Korean
tu Turkish
vi Vietnamese
it Italian
ur Urdu
pl Polish
uk Ukrainian
ro Romanian
az Azerbaijani
ha Hausa
my Burmese
th Thai
nl Dutch
yo Yoruba
id Indonesian
sv Swedish
no Norwegian
he Hebrew
da Danish
ms Malay
fl Filipino
fi Finnish
cs Czech
sk Slovak
hy Armenian
bg Bulgarian
ka Georgian
el Greek
hr Croatian
hu Hungarian
lt Lithuanian
lv Latvian
mk Macedonian
sr Serbian
fr French
ar Arabic
fa Persian
ru Russian

Historical Schedules API

Get the whole airport’s timetable information (either departure or arrival) in 1 API call or track individual flights by using the filters. You can also call all flights of a single airline or filter the flights from or to a specific destination.

Servers

https://aviation-edge.com/v2/public/

Authorize

Historical Schedules API

GET/flightsHistory

Schemas

HistoricalSchedulesResponse

Future Schedules Schedules API

Get airport arrival and departure schedules of airport timetables based on a future date.

Servers

https://aviation-edge.com/v2/public

Authorize

Future Schedules API

GET/flightsFuture

Schemas

FutureSchedulesResponse

Airline Routes API

Filter the data on individual flights, seperate airlines or departure airports to get the aviation routes you need.

Servers

https://aviation-edge.com/v2/public/

Authorize

Airline Routes API

GET/routes

Schemas

AirlineRoutesResponse

Flight Delay API

The Flight Delay API service provides historical flight schedule and timetables data of all airports and airlines around the world.

Servers

https://aviation-edge.com/v2/public/

Authorize

Flight Delay API

GET/flightsHistory

Schemas

FlightDelayResponse

Nearby API

Get all the detailed information based on locations of your choice.

Servers

https://aviation-edge.com/v2/public

Authorize

Nearby Airports and Cities API

GET/nearby

Schemas

NearbyAirportsResponse

Autocomplete API

Using any query, you can get direct output of all the available cities, airports, railway and bus stations and much more. We take care that the data is actual for you.

Servers

https://aviation-edge.com/v2/public

Authorize

Aviation Autocomplete API

GET/autocomplete

Schemas

AutocompleteResponse

Satellite Tracker API

Get all orbiting satellite information in 1 API call or track individual satellites. You can also call all satellites based on the launch year or filter on your end based on the name if you are interested in certain satellites such as Starlink or ISS which are client favorites.

Servers

https://aviation-edge.com/v2/public

Authorize

Satellite Tracker API

GET/satelliteDetails

Schemas

SatelliteTrackerResponse

Static Data

Airports Database

Our Airport Database is an all-in, detailed database of all airports in the world. All internationally used codes such as IATA and ISO codes, names and local translations (where available), longitude and latitude information, country and timezone information, routes, contact detail and many more groups of data are included in this database.

Servers

https://aviation-edge.com/v2/public

Authorize

Airport Database API

GET/airportDatabase

Schemas

AirportsResponse

Airlines Database

The Airlines Database is a complete database of all airlines which include IATA and ICAO-related code, codehub, call signs, country information with country codes, type and status, official name and slug name of airlines, size, fleet and founding year information.

Servers

https://aviation-edge.com/v2/public

Authorize

Airline Database API

GET/airlineDatabase

Schemas

AirlineResponse

Aircraft Database

The Aircraft Database is a very large standalone database that contains publicly available information of many types of aircrafts. The data includes registration information, related ICAO and IATA codes, production information (aircraft brand, series and model, engine type, age and many more), flight data, delivery and first flight dates, status of the aircraft and more.

Servers

https://aviation-edge.com/v2/public

Authorize

Airplane Database API

GET/airplaneDatabase

Schemas

AirplaneResponse

Cities Database

The City Database has data regarding all cities in the world in it. IATA and ISO codes, official names and local translations (where available), related timezone, geographical data, longitude and latitude information and many more can be reached through this standalone database.

Servers

https://aviation-edge.com/v2/public

Authorize

City Database API

GET/cityDatabase

Schemas

CityResponse

Countries Database

Aviation Edge Country Database includes complete data of all countries. This data includes ISO codes,official country name, name and the code of the official currency, continent info, FIPS codes, official languages, population, neighboring countries, local translation of country name, capital city and phone prefixes.

Servers

https://aviation-edge.com/v2/public

Authorize

Country Database API

GET/countryDatabase

Schemas

CountryResponse

Taxes Database

Airline Tax Codes Database is the source for aviation related taxes. It includes IATA codes of taxes and tax names which also explains the nature of the taxes.

Servers

https://aviation-edge.com/v2/public

Authorize

Tax Codes API

GET/taxDatabase

Schemas

TaxCodesResponse

Aviation Edge © 2010-2024 – DMCA Protected – Clearsky LTD

@nickscamara
Copy link
Member

Looking into it!

@nickscamara nickscamara self-assigned this Jan 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants