ArgumentError: (array RequestOptions.errorMessages) Expected values to be of type string but received type null

[gyn7561]

Describe the bug
ArgumentError: (array RequestOptions.errorMessages) Expected values to be of type string but received type null

To Reproduce
sqlite json field
{"id":"KBXoZCDAj5e6ZbO","url":"xxxxxxxxx","uniqueKey":"xxxxxxxxx","method":"GET","noRetry":false,"retryCount":1,"errorMessages":[null],"headers":{},"userData":{"page":1}}

Expected behavior
process exit

System information:

• OS: WIN11
• Node.js version 14
• Apify SDK version 2.3

[B4nan]

Can you provide a reproduction? I mean some code, what you provided is an invalid state, we need to understand how you got to that state.

@metalwarrior665 any idea how this could happen? We could ignore null values there, but that would be more of a workaround than a fix I guess.

[metalwarrior665]

Maybe the user created an Error object and then set the message property to null or some library did this? I never saw this.

[gyn7561]

I remove this sqlite file,and it not happen again.

//departStore is a lmdb instance

const crawler = new Apify.CheerioCrawler({
        maxConcurrency: 100,
        minConcurrency: 10,
        // Let the crawler fetch URLs from our list.
        requestQueue,

        maxRequestRetries: 10000,
        // This function will be called for each URL to crawl.
        // It accepts a single parameter, which is an object with options as:
        // https://sdk.apify.com/docs/typedefs/cheerio-crawler-options#handlepagefunction
        // We use for demonstration only 2 of them:
        // - request: an instance of the Request class with information such as URL and HTTP method
        // - $: the cheerio object containing parsed HTML
        handlePageFunction: async ({ request, $ }) => {
            log.debug(`Processing ${request.url} `);
            log.info(JSON.stringify(await requestQueue.getInfo()));

            let allLinks = await RingoTools.extract.extractAllUrls($, request.loadedUrl);
            let firstPageUrl = request.url;
            if (request.url.includes("?")) {
                firstPageUrl = request.url.split("?")[0];
            }
            let header = await RingoTools.extract.extractAttrByCss($, ["text"], ".z-header");
            let links = allLinks.filter(link => link.startsWith(firstPageUrl + "?first_id="));
            if (header.length === 0) {
                throw new Error("no header info" + request.url);
            }
            if (request.userData && request.userData.type === "first") {
                for (let i = 0; i < links.length; i++) {
                    await requestQueue.addRequest({ url: links[i], userData: { type: "list" } });
                }
            }

            let detailLinks = allLinks.filter(link => link.startsWith("http://3g"));
            let hospitalId = firstPageUrl.replace("http://3g", "").replace("-1.htm", "");
            departStore.putSync(request.url, { hospitalId, detailLinks, html: $.html() });
        },

        // This function is called if the page processing failed more than maxRequestRetries+1 times.
        handleFailedRequestFunction: async ({ request }) => {
            await requestQueue.reclaimRequest(request);
        }
    });

[hursey013]

I'm seeing this issue as well - any suggestions on how to prevent this?

  const crawler = new Apify.BasicCrawler({
    requestQueue,
    handleFailedRequestFunction: async ({ request, error }) => {
      logger.warn('Error processing request.', { error, request })
    },
    handleRequestFunction: async ({ request }) => {
      if (parentPort)
        parentPort.once('message', msg => msg === 'cancel' && cancel())

      logger.info('Processing request...', { request })

      const { trackers } = await handleProducts(
        repo,
        request,
        requestQueue,
        snap.val(),
        token,
        'products',
        trackerCache,
      )

      trackerCache = { ...trackerCache, ...trackers }
    },
  })

  await crawler.run()

  logger.info('Crawler finished.')

2022-07-16T13:16:57.100636249Z app[web.1]: WARN  BasicCrawler: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null` (retrying 1/3)
2022-07-16T13:16:57.102683829Z app[web.1]: WARN  BasicCrawler: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null` (retrying 2/3)
2022-07-16T13:16:57.103109533Z app[web.1]: WARN  BasicCrawler: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null` (retrying 3/3)
2022-07-16T13:16:57.103500991Z app[web.1]: ERROR BasicCrawler:AutoscaledPool: runTaskFunction failed.
2022-07-16T13:16:57.103514737Z app[web.1]:   ArgumentError: (array `RequestOptions.errorMessages`) Expected values to be of type `string` but received type `null`
2022-07-16T13:16:57.103520447Z app[web.1]:       at ow (/app/node_modules/ow/dist/index.js:32:24)
2022-07-16T13:16:57.103524515Z app[web.1]:       at /app/node_modules/apify/build/request.js:112:34
2022-07-16T13:16:57.103528442Z app[web.1]:       at Array.forEach (<anonymous>)
2022-07-16T13:16:57.103532790Z app[web.1]:       at new Request (/app/node_modules/apify/build/request.js:108:30)
2022-07-16T13:16:57.103537509Z app[web.1]:       at RequestQueue.getRequest (/app/node_modules/apify/build/storages/request_queue.js:236:16)
2022-07-16T13:16:57.103542128Z app[web.1]:       at async RequestQueue.fetchNextRequest (/app/node_modules/apify/build/storages/request_queue.js:274:23)
2022-07-16T13:16:57.103546696Z app[web.1]:       at async /app/node_modules/apify/build/crawlers/basic_crawler.js:416:23
2022-07-16T13:16:57.103552387Z app[web.1]:       at async wrap (/app/node_modules/@apify/timeout/index.js:73:27)