Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

on_demand api's exit when provider empties rather than when relying api's are finished #36

Open
tkmcmaster opened this issue Oct 28, 2020 · 0 comments

Comments

@tkmcmaster
Copy link
Contributor

Describe the bug

The test exits early (with no message as to why) when a provider runs out to an on_demand api.

Expected behavior

The test should continue to run until non-on_demand api calls can't get data. Also, it should log why it exited early (provider ran out)

Config file

vars:
  port: ${PORT}
load_pattern:
  - linear:
      from: 100%
      to: 100%
      over: 5m
config:
  client:
    headers:
      Accept: 'application/json'
  general:
    bucket_size: 1m
    log_provider_stats: 1m
providers:
  counterProvider:
    range:
      start: 1
      end: 10
      repeat: false
  response1:
    response: {}
  response2:
    response: {}
endpoints:
  - method: GET
    url: 'http://localhost:${port}?count=${counterProvider}'
    on_demand: true
    peak_load: 60hpm
    provides:
      response1:
        select: counterProvider
        where: response.status <= 300
        send: force
  - method: GET
    url: 'http://localhost:${port}?count=${response1}'
    on_demand: true
    peak_load: 60hpm
    provides:
      response2:
        select: response1
        where: response.status <= 300
        send: force
  - method: GET
    url: 'http://localhost:${port}?count=${response2}'
    peak_load: 60hpm

Command to run

$ PORT=8085 pewpew run ProviderEndsEarly.yaml

System info

  • Operating System: Windows 10
  • Pewpew version: 5.7 and 5.8 preview 1

Additional context

It seems inconsistent. Occasionally I get 10 calls on each api call, but most runs the final API only runs 9 times.

$ PORT=8085 pewpew run ProviderEndsEarly.yaml
Starting load test. Test will end around 10:15:32 28-Oct-2020 in approximately 5 minutes

Bucket Summary 10:10:00 to 10:11:00 28-Oct-2020

- GET http://localhost:8085?count=*:
  calls made: 10
  status counts: {204: 10}
  p50: 0.741ms, p90: 1.439ms, p95: 330.495ms, p99: 330.495ms, p99.9: 330.495ms
  min: 0.459ms, max: 330.495ms, avg: 33.744ms, std. dev: 98.875ms

- GET http://localhost:8085?count=*:
  calls made: 10
  status counts: {204: 10}
  p50: 0.878ms, p90: 1.492ms, p95: 2.133ms, p99: 2.133ms, p99.9: 2.133ms
  min: 0.662ms, max: 2.133ms, avg: 1.051ms, std. dev: 0.413ms

- GET http://localhost:8085?count=*:
  calls made: 9
  status counts: {204: 9}
  p50: 0.877ms, p90: 1.349ms, p95: 1.349ms, p99: 1.349ms, p99.9: 1.349ms
  min: 0.686ms, max: 1.349ms, avg: 0.904ms, std. dev: 0.197ms

Test will end around 10:10:42 28-Oct-2020 in approximately 0 seconds

Test Summary 10:10:32 to 10:15:32 28-Oct-2020

- GET http://localhost:8085?count=*:
  calls made: 10
  status counts: {204: 10}
  p50: 0.741ms, p90: 1.439ms, p95: 330.495ms, p99: 330.495ms, p99.9: 330.495ms
  min: 0.459ms, max: 330.495ms, avg: 33.744ms, std. dev: 98.875ms

- GET http://localhost:8085?count=*:
  calls made: 10
  status counts: {204: 10}
  p50: 0.878ms, p90: 1.492ms, p95: 2.133ms, p99: 2.133ms, p99.9: 2.133ms
  min: 0.662ms, max: 2.133ms, avg: 1.051ms, std. dev: 0.413ms

- GET http://localhost:8085?count=*:
  calls made: 9
  status counts: {204: 9}
  p50: 0.877ms, p90: 1.349ms, p95: 1.349ms, p99: 1.349ms, p99.9: 1.349ms
  min: 0.686ms, max: 1.349ms, avg: 0.904ms, std. dev: 0.197ms
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant