expose to_pandas_kwargs in read_parquet with pyarrow backend
#59654
Conversation
kleinhenz
changed the title
to_pandas_kwargsto_pandas_kwargs in read_parquet with pyarrow backend
|
@jorisvandenbossche would you be able to review? Tagging you since you mentioned this in #56842. |
Co-authored-by: Xiao Yuan <[email protected]>
I wasn't sure if I should do this since it is an engine specific option. Happy to add it though. |
|
Looking forward to upgrading to this fix. Due to this issue, I am currently using pickles, which is not ideal. |
|
This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this. |
|
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen. |
jorisvandenbossche
added
Enhancement
IO Parquet
| @@ -127,7 +127,7 @@ MultiIndex | |||
|
|
|||
| I/O | |||
| ^^^ | |||
| - | |||
| - ``pyarrow`` engine for :func:`read_parquet` accepts ``to_pandas_kwargs`` which are forwarded to :meth:`pyarrow.Table.to_pandas`. This enables passing in ``maps_as_pydicts`` to read parquet map datatypes as python dictionaries. (:issue:`56842`) | |||
There was a problem hiding this comment.
Can you move this to the 3.0.0.rst file? At the moment general new features are targeted for 3.0.0
|
(I see that in #57044, we had some discussion if we actually want this .. Will move that part of the discussion to the issue) |
to_pandas_kwargsinread_parquetfor pyarrow engine #49236, BUG: parquet serialization/deserialization adds all dict keys into column #56842doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.