AbstractInterpreter: refactor the lifetimes of OptimizationState and IRCode

[aviatesk]

This commit limits the lifetimes of OptimizationState and IRCode
for a more dataflow clarity. It also avoids duplicated calls of ir_to_codeinf!.
With this change, for NativeInterpreter, Julia-level optimization-specific
information won't leak to other parts of the compilation pipeline, and
such information gets transformed to cacheable information at the end of
the optimization phase.

Note that external AbstractInterpreters can still extend their
lifetimes to cache additional information, as described by this
newly added documentation of finish!:

finish!(interp::AbstractInterpreter,
    opt::OptimizationState, ir::IRCode, caller::InferenceResult)

Runs post-Julia-level optimization process and caches information for later uses:

• computes "purity" (i.e. side-effect-freeness) of the optimized frame
• computes inlining cost and cache the inlineability in opt.src.inlineable
• stores the result of optimization in caller.src

• by default, caller.src will be an optimized CodeInfo object transformed from ir
• in a case when this frame has been proven pure, ConstAPI object wrapping the constant
  value will be kept in caller.src instead, so that the runtime system will use
  the constant calling convention

!!! note
The lifetimes of opt and ir end by the end of this process.
Still external AbstractInterpreter can override transform_optresult_for_cache
as necessary to cache them. Note that transform_result_for_cache should be overloaded
also in such cases, otherwise the default implmentation of transform_result_for_cache
will discard any information other than CodeInfo, Vector{UInt8} or ConstAPI.

---

This commit also adds a new overload infresult_iterator so that external
interpreters can tweak the behavior of post processings of _typeinf.
Especially, this change is motivated by the need for JET, whose post-optimization
processing needs references of InferenceState.

[aviatesk]

Cthulhu can be updated like this PR: JuliaDebug/Cthulhu.jl#264

[vtjnash]

seems good to me, if it works for you to make things better

[aviatesk]

@nanosoldier runbenchmarks("inference", vs=":master")

[aviatesk]

@nanosoldier runbenchmarks(!"scalar", vs=":master")

[nanosoldier]

Something went wrong when running your job:

NanosoldierError: failed to run benchmarks against primary commit: failed process: Process(`sudo -n /nanosoldier/cset/bin/cset shield -e -- sudo -n -u nanosoldier-worker -- /nanosoldier/workdir/jl_q9V9Lu/benchscript.sh`, ProcessExited(1)) [1]

Logs and partial data can be found here

[nanosoldier]

Something went wrong when running your job:

NanosoldierError: failed to run benchmarks against primary commit: failed process: Process(`sudo -n /nanosoldier/cset/bin/cset shield -e -- sudo -n -u nanosoldier-worker -- /nanosoldier/workdir/jl_wofuk5/benchscript.sh`, ProcessExited(1)) [1]

Logs and partial data can be found here

[aviatesk]

@nanosoldier runbenchmarks(!("scalar" || "inference"), vs=":master")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

[aviatesk]

@nanosoldier runbenchmarks("random", vs=":master")

[nanosoldier]

Your benchmark job has completed - no performance regressions were detected. A full report can be found here.