Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MNIST example does not work on Julia master #13

Open
DilumAluthge opened this issue Feb 28, 2020 · 1 comment
Open

MNIST example does not work on Julia master #13

DilumAluthge opened this issue Feb 28, 2020 · 1 comment

Comments

@DilumAluthge
Copy link
Contributor

DilumAluthge commented Feb 28, 2020 •
edited
Loading

I think the issue is that the manifest uses an old version of CuArrays that does not work on Julia master.

cc: @Keno

julia> cd("ToyFHE.jl/examples/encrypted_mnist")
julia rm("mnist_conv.bson"; force = true)
julia> include("train.jl")
[ Info: Precompiling CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae]
┌ Warning: Assignment to `name` in soft scope is ambiguous because a global variable by the same name exists: `name` will be treated as a new local. Disambiguate by using `local name` to suppress this warning or `global name` to assign to the existing global variable.
└ @ ~/.julia/packages/CUDAdrv/WqkY2/src/error.jl:83
┌ Warning: `@get!(dict, key, default)` at /users/daluthge/.julia/packages/Requires/9Jse8/src/require.jl:11 is deprecated, use `get!(()->default, dict, key)` instead.
│   caller = include(::Function, ::Module, ::String) at Base.jl:380
└ @ Base ./Base.jl:380
ERROR: LoadError: LoadError: syntax: invalid keyword argument syntax "(escape (= threads nthreads))"
Stacktrace:
 [1] top-level scope at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/array.jl:318
 [2] include(::Function, ::Module, ::String) at ./Base.jl:380
 [3] include at ./Base.jl:368 [inlined]
 [4] include(::String) at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/CuArrays.jl:1
 [5] top-level scope at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/CuArrays.jl:22
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:451
 [11] top-level scope at ./none:3
in expression starting at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/array.jl:318
in expression starting at /users/daluthge/.julia/packages/CuArrays/dY5ry/src/CuArrays.jl:22
ERROR: LoadError: Failed to precompile CuArrays [3a865a2d-5b23-5a0f-bc46-62713ec82fae] to /users/daluthge/.julia/compiled/v1.5/CuArrays/7YFE0_YKowg.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1288
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::String) at ./client.jl:441
 [7] top-level scope at REPL[10]:1
in expression starting at /gpfs_home/daluthge/Desktop/ToyFHE.jl/examples/encrypted_mnist/train.jl:9

julia> versioninfo(verbose = true)
Julia Version 1.5.0-DEV.274
Commit 8eb0f9fefb (2020-02-15 12:41 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
      "Red Hat Enterprise Linux Server release 7.3 (Maipo)"
  uname: Linux 3.10.0-957.5.1.el7.x86_64 #1 SMP Wed Dec 19 10:46:58 EST 2018 x86_64 x86_64
  CPU: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz:
              speed         user         nice          sys         idle          irq
       #1  3601 MHz  302347066 s        170 s   53586390 s  337540460 s          0 s
       #2  3601 MHz  235401396 s        383 s   47128775 s  410325724 s          0 s
       #3  3601 MHz  103237285 s        288 s   30844130 s  559882062 s          0 s
       #4  3601 MHz   63412689 s        364 s   22978592 s  607626432 s          0 s
       #5  3601 MHz  315443735 s        113 s   49256804 s  329997494 s          0 s
       #6  3601 MHz  155708333 s        578 s   37188177 s  501770058 s          0 s
       #7  3601 MHz   54762847 s        546 s   15693504 s  624338616 s          0 s
       #8  3601 MHz   46859843 s        550 s   14616044 s  632741590 s          0 s

  Memory: 93.04103088378906 GB (74548.9140625 MB free)
  Uptime: 6.958699e6 sec
  Load Avg:  1.20654296875  1.44921875  1.22314453125
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  CPLUS_INCLUDE_PATH = /gpfs/runtime/opt/gcc/8.3/include
  MANPATH = /gpfs/runtime/opt/python/3.7.4/share/man:/gpfs/runtime/opt/git/2.20.2/share/man:/gpfs/runtime/opt/gcc/8.3/share/man:/gpfs/runtime/opt/binutils/2.31/share/man:/gpfs/runtime/opt/intel/2017.0/man/common/man1:
  TERM = xterm-256color
  LIBRARY_PATH = /gpfs/runtime/opt/cuda/10.2/cuda/lib64:/gpfs/runtime/opt/cuda/10.2/cuda/lib:/gpfs/runtime/opt/cudnn/7.6.5/lib64:/gpfs/runtime/opt/python/3.7.4/lib:/gpfs/runtime/opt/binutils/2.31/lib:/gpfs/runtime/opt/intel/2017.0/lib/intel64:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64
  INCLUDE_PATH = /gpfs/runtime/opt/cudnn/7.6.5/include
  CUDA_HOME = /gpfs/runtime/opt/cuda/10.2/cuda
  LD_LIBRARY_PATH = /gpfs/runtime/opt/cuda/10.2/src/lib64:/gpfs/runtime/opt/cuda/10.2/src/lib:/gpfs/runtime/opt/cudnn/7.6.5/lib64:/gpfs/runtime/opt/python/3.7.4/lib:/gpfs/runtime/opt/gcc/8.3/lib64:/gpfs/runtime/opt/binutils/2.31/lib:/gpfs/runtime/opt/intel/2017.0/lib/intel64:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64:/gpfs/runtime/opt/java/8u111/jre/lib/amd64
  CPATH = /gpfs/runtime/opt/cuda/10.2/cuda/include:/gpfs/runtime/opt/cudnn/7.6.5/include:/gpfs/runtime/opt/python/3.7.4/include:/gpfs/runtime/opt/gcc/8.3/include:/gpfs/runtime/opt/binutils/2.31/include:/gpfs/runtime/opt/intel/2017.0/mkl/include
  NLSPATH = /gpfs/runtime/opt/intel/2017.0/lib/intel64/locale/en_US:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64/locale/en_US
  PATH = /gpfs/runtime/opt/cuda/10.2/cuda/bin:/users/daluthge/bin:/gpfs/runtime/opt/python/3.7.4/bin:/gpfs/runtime/opt/git/2.20.2/bin:/gpfs/runtime/opt/gcc/8.3/bin:/gpfs/runtime/opt/binutils/2.31/bin:/gpfs/runtime/opt/intel/2017.0/bin:/gpfs/runtime/opt/matlab/R2017b/bin:/gpfs/runtime/opt/java/8u111/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/lpp/mmfs/bin:/usr/lpp/mmfs/sbin:/opt/ibutils/bin:/gpfs/runtime/bin
  C_INCLUDE_PATH = /gpfs/runtime/opt/cudnn/7.6.5/include:/gpfs/runtime/opt/gcc/8.3/include
  LD_RUN_PATH = /gpfs/runtime/opt/cuda/10.2/cuda/lib64:/gpfs/runtime/opt/cuda/10.2/cuda/lib:/gpfs/runtime/opt/cudnn/7.6.5/lib64:/gpfs/runtime/opt/cudnn/7.6.5/lib:/gpfs/runtime/opt/python/3.7.4/lib:/gpfs/runtime/opt/gcc/8.3/lib64:/gpfs/runtime/opt/binutils/2.31/lib:/gpfs/runtime/opt/intel/2017.0/lib/intel64:/gpfs/runtime/opt/intel/2017.0/mkl/lib/intel64
  JAVA_HOME = /gpfs/runtime/opt/java/8u111
  MODULEPATH = /gpfs/runtime/modulefiles
  HOME = /users/daluthge
  IPP_PATH = /gpfs/runtime/opt/intel/2017.0/ipp
  MODULEHOME = /gpfs/runtime/pymodules
  PKG_CONFIG_PATH = /gpfs/runtime/opt/python/3.7.4/lib/pkgconfig
  QT_PLUGIN_PATH = /usr/lib64/kde4/plugins:/usr/lib/kde4/plugins

(ToyFHE) pkg> st
Project ToyFHE v0.1.0
Status `/gpfs_home/daluthge/Desktop/ToyFHE.jl/Project.toml`
  [c3fe647b] AbstractAlgebra v0.7.1 #master (https://github.com/Nemocas/AbstractAlgebra.jl.git)
  [15f4f7f2] AutoHashEquals v0.2.0
  [fbb218c0] BSON v0.2.4
  [c3b6d118] BitIntegers v0.2.0 #kf/div (https://github.com/Keno/BitIntegers.jl)
  [3a865a2d] CuArrays v1.3.0 #master (https://github.com/JuliaGPU/CuArrays.jl.git)
  [ab62b9b5] DeepDiffs v1.1.0
  [31c24e10] Distributions v0.21.1 #kf/discretenormal (https://github.com/Keno/Distributions.jl)
  [7a1cc6ca] FFTW v1.0.1
  [587475ba] Flux v0.9.0
  [65ccaadd] FourierTransforms v0.0.0 #master (https://github.com/JuliaComputing/FourierTransforms.jl)
  [0c68f7d7] GPUArrays v1.0.4 #master (https://github.com/JuliaGPU/GPUArrays.jl.git)
  [8d0d7f98] GaloisFields v0.4.0 #master (https://github.com/tkluck/GaloisFields.jl.git)
  [3e1990a7] Hecke v0.6.5 #kf/discard2 (https://github.com/Keno/Hecke.jl)
  [7869d1d1] IRTools v0.3.0 #master (https://github.com/MikeInnes/IRTools.jl.git)
  [aa1ae85d] JuliaInterpreter v0.7.4
  [7475f97c] Mods v0.1.0
  [2edaba10] Nemo v0.15.1 #master (https://github.com/Nemocas/Nemo.jl.git)
  [6fe1bfb0] OffsetArrays v0.11.1
  [f27b6e38] Polynomials v0.5.3
  [27ebfcd6] Primes v0.4.0+ #master (https://github.com/JuliaMath/Primes.jl.git)
  [09ab397b] StructArrays v0.4.0 #kf/fhe (https://github.com/Keno/StructArrays.jl)
  [700de1a5] ZygoteRules v0.2.0 #master (https://github.com/FluxML/ZygoteRules.jl.git)
  [37e2e46d] LinearAlgebra
  [9a3f8284] Random

(ToyFHE) pkg> st -m
Project ToyFHE v0.1.0
Status `/gpfs_home/daluthge/Desktop/ToyFHE.jl/Manifest.toml`
  [c3fe647b] AbstractAlgebra v0.7.1 #master (https://github.com/Nemocas/AbstractAlgebra.jl.git)
  [621f4979] AbstractFFTs v0.4.1
  [1520ce14] AbstractTrees v0.2.1
  [79e6a3ab] Adapt v1.0.0
  [7d9fca2a] Arpack v0.3.1
  [15f4f7f2] AutoHashEquals v0.2.0
  [fbb218c0] BSON v0.2.4
  [9e28174c] BinDeps v0.8.10
  [b99e7846] BinaryProvider v0.5.8
  [c3b6d118] BitIntegers v0.2.0 #kf/div (https://github.com/Keno/BitIntegers.jl)
  [fa961155] CEnum v0.2.0
  [3895d2a7] CUDAapi v1.2.0
  [c5f51814] CUDAdrv v4.0.3
  [be33ccc6] CUDAnative v2.5.4
  [da1fd8a2] CodeTracking v0.5.8
  [944b1d66] CodecZlib v0.6.0
  [3da002f7] ColorTypes v0.8.0
  [5ae59095] Colors v0.9.6
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v2.2.0
  [8f4d0f93] Conda v1.3.0
  [3a865a2d] CuArrays v1.3.0 #master (https://github.com/JuliaGPU/CuArrays.jl.git)
  [9a962f9c] DataAPI v1.1.0
  [864edb3b] DataStructures v0.17.5
  [ab62b9b5] DeepDiffs v1.1.0
  [163ba53b] DiffResults v0.0.4
  [b552c78f] DiffRules v0.0.10
  [31c24e10] Distributions v0.21.1 #kf/discretenormal (https://github.com/Keno/Distributions.jl)
  [7a1cc6ca] FFTW v1.0.1
  [1a297f60] FillArrays v0.7.4
  [53c48c17] FixedPointNumbers v0.6.1
  [587475ba] Flux v0.9.0
  [f6369f11] ForwardDiff v0.10.6
  [65ccaadd] FourierTransforms v0.0.0 #master (https://github.com/JuliaComputing/FourierTransforms.jl)
  [0c68f7d7] GPUArrays v1.0.4 #master (https://github.com/JuliaGPU/GPUArrays.jl.git)
  [8d0d7f98] GaloisFields v0.4.0 #master (https://github.com/tkluck/GaloisFields.jl.git)
  [cd3eb016] HTTP v0.8.8
  [3e1990a7] Hecke v0.6.5 #kf/discard2 (https://github.com/Keno/Hecke.jl)
  [7869d1d1] IRTools v0.3.0 #master (https://github.com/MikeInnes/IRTools.jl.git)
  [83e8ac13] IniFile v0.5.0
  [682c06a0] JSON v0.21.0
  [aa1ae85d] JuliaInterpreter v0.7.4
  [e5e0dc1b] Juno v0.7.2
  [929cbde3] LLVM v1.3.2
  [1914dd2f] MacroTools v0.5.2
  [739be429] MbedTLS v0.7.0
  [e89f7d12] Media v0.5.0
  [e1d29d7a] Missings v0.4.3
  [7475f97c] Mods v0.1.0
  [872c559c] NNlib v0.6.0
  [77ba4419] NaNMath v0.3.2
  [2edaba10] Nemo v0.15.1 #master (https://github.com/Nemocas/Nemo.jl.git)
  [6fe1bfb0] OffsetArrays v0.11.1
  [bac558e1] OrderedCollections v1.1.0
  [90014a1f] PDMats v0.9.10
  [69de0a69] Parsers v0.3.8
  [f27b6e38] Polynomials v0.5.3
  [2dfb63ee] PooledArrays v0.5.2
  [27ebfcd6] Primes v0.4.0+ #master (https://github.com/JuliaMath/Primes.jl.git)
  [1fd47b50] QuadGK v2.1.1
  [3cdcf5f2] RecipesBase v0.7.0
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v0.5.2
  [79098fc4] Rmath v0.6.0
  [a2af1166] SortingAlgorithms v0.3.1
  [276daf66] SpecialFunctions v0.7.2
  [90137ffa] StaticArrays v0.12.1
  [2913bbd2] StatsBase v0.32.0
  [4c63d2b9] StatsFuns v0.8.0
  [09ab397b] StructArrays v0.4.0 #kf/fhe (https://github.com/Keno/StructArrays.jl)
  [a759f4b9] TimerOutputs v0.5.3
  [9f7883ad] Tracker v0.2.5
  [3bb67fe8] TranscodingStreams v0.9.5
  [30578b45] URIParser v0.4.0
  [81def892] VersionParsing v1.1.3
  [a5390f91] ZipFile v0.8.3
  [700de1a5] ZygoteRules v0.2.0 #master (https://github.com/FluxML/ZygoteRules.jl.git)
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8bb1440f] DelimitedFiles
  [8ba89e20] Distributed
  [b77e0a4c] InteractiveUtils
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [a63ad114] Mmap
  [44cfe95a] Pkg
  [de0858da] Printf
  [9abbd945] Profile
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [1a1011a3] SharedArrays
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [4607b0f0] SuiteSparse
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
@Keno
Copy link
Collaborator

Keno commented Feb 28, 2020

I can look into updating the manifest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Keno @DilumAluthge