Skip to content

Remove imports from black_scholes - #165

Closed
Hardcode84 wants to merge 25 commits into
IntelPython:mainfrom
Hardcode84:nuke-imports
Closed

Hardcode84 wants to merge 25 commits into
IntelPython:mainfrom
Hardcode84:nuke-imports

Conversation

@Hardcode84

@Hardcode84 Hardcode84 commented Nov 7, 2022 •

Copy link
Copy Markdown
Contributor
  • Instead of importing all implementations in __init__.py, import them dynamically using importlib.import_module
  • numpy and python impls are still needed to be imported explicitly as reference implementation checking code wasn't adapted to new implm done
  • target=tout.exit_after(timeout)(_exec), was failing on my py 3.9 win machine with AttributeError: Can't pickle local object 'exit_after.<locals>.outer.<locals>.inner', need to investigate
  • os.remove(output_npz) was failing with The process cannot access the file because it is being used by another process

@Hardcode84
Hardcode84 requested a review from diptorupd November 7, 2022 21:17
@Hardcode84 Hardcode84 changed the title [WIP] Remove imports from black_scholes Remove imports from black_scholes Nov 20, 2022
@Hardcode84
Hardcode84 marked this pull request as ready for review November 20, 2022 16:52
@Hardcode84

Hardcode84 commented Nov 20, 2022 •

Copy link
Copy Markdown
Contributor Author

@diptorupd I think it is ready for review. If this approach is ok, I'll update rest of the workloads in followup PRs. But, before this, can you please fix the CI so it can actually report errors.

Thanks

@mingjie-intel

Copy link
Copy Markdown
Contributor

The changes failed the single black scholes run:

For example:

python -c "import dpbench; dpbench.run_benchmark(\"black_scholes\")"


================ Benchmark black_scholes ========================

Please take a look.

@Hardcode84

Copy link
Copy Markdown
Contributor Author

The changes failed the single black scholes run:

This should be fixed now.

Added all_benchmarks field to __init__.py, so module can actually report list of supported benchmarks without importing them. We can go without it completely and instead just try to import all modules based on list of known postfixes, but this require module name to follow naming convention exactly and a bit fragile.

@mingjie-intel

Copy link
Copy Markdown
Contributor

It seems fixed the most of the issue. The hanging seems going away. One thing is showing up after fix is somehow sycl test is disabled, as seen here:

================ implementation sycl ========================
implementation: sycl
error states: ErrorCodes.UNIMPLEMENTED
error msg: No implementation

@mingjie-intel

Copy link
Copy Markdown
Contributor

It seems the logic of running dpbench tests is broken. All the cases didn't launch correctly after black scholes. Here is an example for error message:

ERROR:root:Benchmark execution failed due to the following error: 
Traceback (most recent call last):
  File "/home/runner/work/dpbench/dpbench/dpbench/runner.py", line 101, in run_benchmark
    results = bench.run(
  File "/home/runner/work/dpbench/dpbench/dpbench/infrastructure/benchmark.py", line 943, in run
    runner = BenchmarkRunner(
  File "/home/runner/work/dpbench/dpbench/dpbench/infrastructure/benchmark.py", line 440, in __init__
    self.impl_fn = self.bench.get_impl(impl_postfix)
  File "/home/runner/work/dpbench/dpbench/dpbench/infrastructure/benchmark.py", line 843, in get_impl
    impl_fnlist = self._get_updated_fnlist(impl_postfix, self.impl_fnlist)
  File "/home/runner/work/dpbench/dpbench/dpbench/infrastructure/benchmark.py", line 836, in _get_updated_fnlist
    return impl_fnlist + [(name, getattr(mod, self.bname))]
AttributeError: module 'dpbench.benchmarks.black_scholes.black_scholes_sycl_native_ext' has no attribute 'black_scholes'
ERROR:root:Skipping the benchmark execution due to the following error: 

@Hardcode84

Copy link
Copy Markdown
Contributor Author

BTW, I'm seeing numba-0.54.1 in CI logs for some reason

@Hardcode84

Copy link
Copy Markdown
Contributor Author

sycl impl just refuses to cooperate, will look into it more tomorrow

@Hardcode84

Copy link
Copy Markdown
Contributor Author

Seems to work now, but

  • Had to import sycl impl using full path like black_scholes_sycl_native_ext.black_scholes_sycl._black_scholes_sycl, otherwise it just refused to find attributes for some reason
  • Error output mixes into random places into benchmark output, probably because one is using logger and other is just usual python print

@Hardcode84

Copy link
Copy Markdown
Contributor Author

Can we get this merged?

@Hardcode84

Copy link
Copy Markdown
Contributor Author

@diptorupd @mingjie-intel can we merge this?

@diptorupd
diptorupd requested a review from mingjie-intel March 23, 2023 06:53
@mingjie-intel

Copy link
Copy Markdown
Contributor

Some comments here:

  1. Changes in benchmark.py will not resolve hanging issue on dGPU. I suggest the code modified related can be reverted for now until we have fix for multi-process hanging issue.
  2. Similar changes should be made for all benchmarks. This can be done within this PR or a separate PR. What is your preference? @diptorupd

@adarshyoga

Copy link
Copy Markdown
Contributor
  1. Similar changes should be made for all benchmarks. This can be done within this PR or a separate PR. What is your preference? @diptorupd

Lets make similar changes to all benchmarks in this PR so that we have all the changes before merge this.

@mingjie-intel mingjie-intel mentioned this pull request Mar 23, 2023
1 of 5 tasks
@diptorupd

Copy link
Copy Markdown
  1. Similar changes should be made for all benchmarks. This can be done within this PR or a separate PR. What is your preference? @diptorupd

Lets make similar changes to all benchmarks in this PR so that we have all the changes before merge this.

New PRs please. I will like to get this PR merged as-is if possible.

@diptorupd

Copy link
Copy Markdown
1. Changes in benchmark.py will not resolve hanging issue on dGPU. I suggest the code modified related can be reverted for now until we have fix for multi-process hanging issue.

I agree.

results_dict = manager.dict()
p = Process(
target=tout.exit_after(timeout)(_exec),
target=_exec,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert this and update in #166 if needed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without it it fails on my win machine in 100% cases so I can't test it.

)
p.start()
res = p.join(timeout * 1.2)
res = p.join(timeout)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert this

{outarr: npzfile[outarr]}
)
os.remove(output_npz)
try:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without it it fails on my win machine in 100% cases so I can't test it.

@ZzEeKkAa ZzEeKkAa mentioned this pull request Mar 23, 2023
4 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants