Tensorflow系列一:环境搭建

xiaoxiao2021-02-28  102

这个系列主要摘自《Tensorflow实战Google深度学习框架》这本书,边学习边整理其中主要内容,以便记忆。

Tensorflow系列一环境搭建 一 Tensorflow的主要依赖包 Protocol BufferBazel 二 Tensorflow安装 使用 Virtualenv安装使用native pip安装使用Docker安装使用Anaconda安装从源码编译安装 三 Tensorflow测试样例

Tensorflow系列一:环境搭建

一 Tensorflow的主要依赖包

1. Protocol Buffer

​ Protocol Buffer是谷歌开发的处理结构化数据的工具。这里结构化数据指的是拥有多种属性的数据。假设用户信息包括名字,ID和Email地址。

一个用户的信息表示示例name:张三id: 12345email: zhangsan@abc.com

​ 当要将结构化数据数据持久化或进行网络传输时,需要先将它序列化(将结构化数据变成数据流的格式,简单地说就是变为一个字符串)。如何将结构化数据序列化,并从序列化数据流中还原结构化数据,称为处理结构化数据。

​ XML和JSON是两种常用的结构化数据处理工具。

<!-- xml格式示例 --> <user> <name>张三</name> <id>12345</id> <email>zhangsan@abc.com</email> </user> //JSON格式示例 { "name":"张三""id":"12345", "email":"zhangsan@abc.com", }

​ 不同于XML和JSON,Protocol Buffer序列化后得到的数据不是可读的字符串威视二进制流。再者Protocol Buffer需要先定义数据的格式(schema)再还原结构化数据。(Protocol Buffer序列化的数据比XML格式数据小3到10倍,解析时间要快20到100倍)。

//Protocol Buffer数据格式定义示例 message user{ optional string name = 1;//optional,取值可为空 required int32 id = 2;//required,每个message实例必须包含该属性 repeated string email = 3;//repeated,属性取值可以是一个列表 }

2. Bazel

​ Bazel是从谷歌开源的自动化构建工具,用于编译应用。

​ 项目空间(worksapce): Bazel的基本概念,包含了编译一个软件所需要的源代码以及输出编译结果的软连接(symbolic link)地址。项目空间对应的文件夹是项目的根目录,需要有:

​ WORKSPACE文件:定义了对外部资源的依赖关系,可为空文件;

​ BULID文件:通过它找到需要编译的目标,该文件中指定可每一个编译目标的输入,输出和编译方式。Bazel的编译方式是事先定义好的,比如Python支持的编译方式只有三种:py_binary(将Python程序编译为可执行文件), py_library(将Python编译为库函数供供py_test或py_binary调用)和py_test(编译Python测试程序)。

​ 下面使用一个简单样例来说明Bazel的工作,如下项目空间有4个文件:

-rw-rw-r-- root root 208 BULID -rw-rw-r-- root root 48 hello_lib.py -rw-rw-r-- root root 47 hell0_main.py -rw-rw-r-- root root 0 WORKSPACE #详细内容如下: #hellp_lib.py def print_hello_world(): print("Hello World") #hello_main.py import hello_lib hello_lib.print_hello_world() #BULID ''' name属性:编译目标名字; srcs属性:编译所需要的源代码,可以是一个列表; deps属性:编译所需依赖关系 ''' py_library( name = "hello_lib", srcs = [ "hello_lib.py" ] ) py_binary( name = "hello_main", srcs = [ "hello_main.py", ], deps = [ ":hello_lib", ], )

​ 在这个项目空间中允许编译操作bazel build :hello_main 将得到编译结果,他们都是通过软连接的形式放在当前的项目空间里,实际编译结果文件保存在~/.cache/bazel目录下,其中bazel-bin目录下寻访了编译产生的二进制文件以及运行该文件所需的里来关系,在当前目录下运行bazel-bin/hello_main实现hello_main.py程序,即在屏幕输出”Hello World“。

二 Tensorflow安装

​ 这部分不再参考书籍内容,而是翻译官网安装文档https://www.tensorflow.org/install/install_linux(installing Tensorflow on Ubuntu)。

1.使用 Virtualenv安装

#安装pip和virtualenv $ sudo apt-get install python-pip python-dev python-virtualenv #创建一个virtualenv环境 $ virtualenv --system-site-packages 目标目录(假设为~/tensorflow) #激活virtualenv环境(以下两个命令之一) $ source ~/tensorflow/bin/activate # bash, sh, ksh, or zsh $ source ~/tensorflow/bin/activate.csh # csh or tcsh #安装Tensorflow(以下命令按所需选择) (tensorflow)$ pip install --upgrade tensorflow # for Python 2.7 (tensorflow)$ pip3 install --upgrade tensorflow # for Python 3.n (tensorflow)$ pip install --upgrade tensorflow-gpu # for Python 2.7 and GPU (tensorflow)$ pip3 install --upgrade tensorflow-gpu # for Python 3.n and GPU #如果上一步安装失败,安装下面的命令之一安装 (tensorflow)$ pip install --upgrade tenfsorflow安装包路径 # Python 2.7 (tensorflow)$ pip3 install --upgrade tenfsorflow安装包路径 # Python 3.N #每次使用Tensorflow前需要激活virtualenv环境(如上),使用完之后可通过以下命令使之失效 (tensorflow)$ deactivate #卸载Tensorflow $ rm -r 目标目录

2.使用”native“ pip安装

#确认python和pip是否已安装 $ python -V $ pip -V #或 pip3 -V #安装pip $ sudo apt-get install python-pip python-dev #安装Tensorflow(选以下命令之一) $ sudo pip install tensorflow # Python 2.7; CPU support (no GPU support) $ sudo pip3 install tensorflow # Python 3.n; CPU support (no GPU support) $ sudo pip install tensorflow-gpu # Python 2.7; GPU support $ sudo pip3 install tensorflow-gpu # Python 3.n; GPU support #如果上一步安装失败,安装下面的命令之一安装 $ sudo pip install --upgrade tenfsorflow安装包路径 # Python 2.7 $ sudo pip3 install --upgrade tenfsorflow安装包路径 # Python 3.N #卸载Tensorflow $ sudo pip uninstall tensorflow # for Python 2.7 $ sudo pip3 uninstall tensorflow # for Python 3.n

3.使用Docker安装

#先安装Docker(若要安装支持GPUs的Tensorflow,需要安装nvidia-docker) #可以创建一个Linux docker group,调用docker可不使用sudo #运行一个包含Tensorflow镜像的docker容器(cpu) $ docker run -it -p hostPort:containerPort Tensorflow镜像 #-p hostPort:containerPort是可选的,-p 8888:8888使用Jupyter Notebooks; -p 6006:6006 使用TensorBoard #Tensorflow镜像(以下按所需选择) #gcr.io/tensorflow/tensorflow, which is the TensorFlow CPU binary image. #gcr.io/tensorflow/tensorflow:latest-devel, which is the latest TensorFlow CPU Binary image plus source code. #gcr.io/tensorflow/tensorflow:version, which is the specified version (for example, 1.0.1) of TensorFlow CPU binary image. #gcr.io/tensorflow/tensorflow:version-devel #GPU $ nvidia-docker run -it -p hostPort:containerPort Tensorflow镜像 #Tensorflow镜像(以下按所需选择) #gcr.io/tensorflow/tensorflow:lastest-gpu, which is the TensorFlow CPU binary image. #gcr.io/tensorflow/tensorflow:latest-devel-gpu, which is the latest TensorFlow CPU Binary image plus source code. #gcr.io/tensorflow/tensorflow:version-gpu, which is the specified version (for example, 1.0.1) of TensorFlow CPU binary image. #gcr.io/tensorflow/tensorflow:version-devel-gpu

4.使用Anaconda安装

#先安装Anaconda #创建一个conda环境命名为tensorflow $ conda create -n tensorflow #激活 $ source activate tensorflow #安装Tensorflow (tensorflow)$ pip install --ignore-installed --upgrade tenfsorflow安装包路径

5.从源码编译安装

#安装Tensorflow依赖的工具包 #1.Bazel(https://bazel.build/versions/master/docs/install-ubuntu.html#install-with-installer-ubuntu) echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add - sudo apt-get update && sudo apt-get install bazel sudo apt-get upgrade bazel#更新Bazel #2.Python依赖包(numpy,dev,pip,wheel) $ sudo apt-get install python-numpy python-dev python-pip python-wheel #for Python 2.7 $ sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel #for Python 3.n #3.可选的:GPU支持 #Nvidia计算能力大于3.0的GPU #Nvidia的Cuda Toolkit(>=7.0)和cuDNN(>=v3) #安装libcupti-dev $ sudo apt-get install libcupti-dev #配置Tensoflow编译环境 #下载最新的Tensorflow源代码 $ git clone https://github.com/tensorflow/tensorflow $ cd tensorflow $ ./configure #配置Python的路径 Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python2.7 Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: Do you wish to use jemalloc as the malloc implementation? [Y/n] jemalloc enabled #配置是否支持谷歌云平台 Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] No Google Cloud Platform support will be enabled for TensorFlow #配置是否支持Hadoop File System Do you wish to build TensorFlow with Hadoop File System support? [y/N] No Hadoop File System support will be enabled for TensorFlow #配置是否支持XLA just-in-time编译 Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N] No XLA JIT support will be enabled for TensorFlow Found possible Python library paths: /usr/local/lib/python2.7/dist-packages /usr/lib/python2.7/dist-packages #配置Python库路径 Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages] Using python library path: /usr/local/lib/python2.7/dist-packages #配置是否支持OpenCL Do you wish to build TensorFlow with OpenCL support? [y/N] N No OpenCL support will be enabled for TensorFlow #配置是否支持CUDA Do you wish to build TensorFlow with CUDA support? [y/N] Y CUDA support will be enabled for TensorFlow #配置GPU Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: #配置 Cuda SDK版本 Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0 #配置Cuda toolkit目录 Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: #配置 cuDnn版本 Please specify the cuDNN version you want to use. [Leave empty to use system default]: 5 #配置cuDnn目录 Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:、 #配置GPU计算能力 Please specify a list of comma-separated Cuda compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size. [Default is: "3.5,5.2"]: 3.0 Setting up Cuda include Setting up Cuda lib Setting up Cuda bin Setting up Cuda nvvm Setting up CUPTI include Setting up CUPTI lib64 Configuration finished #使用Bazel编译pip的安装包 $ bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package #cpu $ bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package #for gpu $ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg #安装 $ sudo pip install /tmp/tensorflow_pkg/tensorflow-1.1.0-py2-none-any.whl

三 Tensorflow测试样例

#打开终端,输入python,使用Tensorflow实现两个向量求和 $ python >>> import tensorflow as tf >>> a = tf.constant([1.0,2.0],name="a") >>> b = tf.constant([2.0,3.0],name="b") >>> c = a + b >>> sess = tf.Session() >>> sess.run(c)
转载请注明原文地址: https://www.6miu.com/read-47379.html

最新回复(0)