caffe中solver不同优化器的一些使用方法(只记录一些常用的) 下面是一些公用的参数 测试时需要前向传播的次数,比如你有1000个数据,批处理大小为10,那么这个值就应该是100,这样才能够将所有的数据覆盖 test_iter: 100 每多少次迭代进行一次测试. test_interval: 500 weight_decay防止过拟合的参数,使用方式: 1 样本越多,该值越小 2 模型参数越多,该值越大 weight_decay: 0.0005
rmsprop: net: "examples/mnist/lenet_train_test.prototxt" test_iter: 100 test_interval: 500 #The base learning rate, momentum and the weight decay of the network. base_lr: 0.01 momentum: 0.0 weight_decay: 0.0005 #The learning rate policy lr_policy: "inv" gamma: 0.0001 power: 0.75 display: 100 max_iter: 10000 snapshot: 5000 snapshot_prefix: "examples/mnist/lenet_rmsprop" solver_mode: GPU type: "RMSProp" rms_decay: 0.98 Adam: net: "examples/mnist/lenet_train_test.prototxt" test_iter: 100 test_interval: 500 #All parameters are from the cited paper above base_lr: 0.001 momentum: 0.9 momentum2: 0.999 #since Adam dynamically changes the learning rate, we set the base learning #rate to a fixed value lr_policy: "fixed" display: 100 #The maximum number of iterations max_iter: 10000 snapshot: 5000 snapshot_prefix: "examples/mnist/lenet" type: "Adam" solver_mode: GPU multistep: net: "examples/mnist/lenet_train_test.prototxt" test_iter: 100 test_interval: 500 #The base learning rate, momentum and the weight decay of the network. base_lr: 0.01 momentum: 0.9 weight_decay: 0.0005 #The learning rate policy lr_policy: "multistep" gamma: 0.9 stepvalue: 5000 stepvalue: 7000 stepvalue: 8000 stepvalue: 9000 stepvalue: 9500 # Display every 100 iterations display: 100 #The maximum number of iterations max_iter: 10000 #snapshot intermediate results snapshot: 5000 snapshot_prefix: "examples/mnist/lenet_multistep" #solver mode: CPU or GPU solver_mode: GPU