Python多线程资源共享锁

本文将简单介绍多线程编程中的线程间资源共享和常用的锁机制。

在多线程编程中，常常会涉及到线程间的资源共享, 常用资源共享常用方式：

全局变量(global)queue(from queue import Queue)

常用的资源共享锁机制：

LockRLockSemphoreCondition

(一) 线程间资源共享

使用全局变量可以实现线程间的资源共享，关键字global

代码演示：

from threading import Thread, Lock lock = Lock() total = 0 '''如果不使用lock那么，最后得到的数字不一定为0；同时loack不支持连续多次acquire，如果这样做了的后果是死锁！''' def add(): global total global lock for i in range(1000000): lock.acquire() total += 1 lock.release() def sub(): global total global lock for i in range(1000000): lock.acquire() total -= 1 lock.release() thread1 = Thread(target=add) thread2 = Thread(target=sub) # 将Thread1和2设置为守护线程，主线程完成时，子线程也一起结束 # thread1.setDaemon(True) # thread1.setDaemon(True) # 启动线程 thread1.start() thread2.start() # 阻塞，等待线程1和2完成，如果不使用join，那么主线程完成后，子线程也会自动关闭。 thread1.join() thread2.join() total 使用queue共享资源，queue是线程安全的。 from threading import Thread, Lock from queue import Queue def add(q): if q.not_full: q.put(1) def sub(q): if q.not_empty: recv = q.get() print(recv) q.task_done() if __name__ =='__main__': # 设置q最多接收3个任务,Queue是线程安全的，所以不需要Lock qu = Queue(3) thread1 = Thread(target=add, args=(qu,)) thread2 = Thread(target=sub, args=(qu,)) thread1.start() thread2.start() # q队列堵塞，等待所有任务都被处理完。 qu.join()

(二) 锁(Lock/RLock/Condition/Semphore)

Lock Lock 不能连续acquire锁，不然会死锁，Lock 资源竞争可能会导致死锁。Lock 会降低性能。 from threading import Thread, Lock lock = Lock() total = 0 '''如果不使用lock那么，最后得到的数字不一定为0；同时lock不支持连续多次acquire，如果这样做了的后果是死锁！''' def add(): global total global lock for i in range(1000000): lock.acquire() total += 1 lock.release() def sub(): global total global lock for i in range(1000000): lock.acquire() total -= 1 lock.release() thread1 = Thread(target=add) thread2 = Thread(target=sub) # 将Thread1和2设置为守护线程，主线程完成时，子线程也一起结束 # thread1.setDaemon(True) # thread1.setDaemon(True) # 启动线程 thread1.start() thread2.start() # 阻塞，等待线程1和2完成，如果不使用join，那么主线程完成后，子线程也会自动关闭。 thread1.join() thread2.join() total RLock RLock 可以连续acquire锁，但是需要相应数量的release释放锁因可以连续获取锁，所以实现了函数内部调用带锁的函数 from threading import Thread, Lock, RLock lock = RLock() total = 0 def add(): global lock global total # RLock实现连续获取锁，但是需要相应数量的release来释放资源 for i in range(1000000): lock.acquire() lock.acquire() total += 1 lock.release() lock.release() def sub(): global lock global total for i in range(1000000): lock.acquire() total -= 1 lock.release() thread1 = Thread(target=add) thread2 = Thread(target=sub) thread1.start() thread2.start() thread1.join() thread2.join() total Condition 条件变量 Condition条件变量服从上下文管理协议：使用with语句获取封闭块持续时间的关联锁。wait（）方法释放锁，然后阻塞，直到另一个线程通过调用notify（）或notify_all（）唤醒它。一旦被唤醒，wait（）重新获得锁并返回。也可以指定超时。先启动wait接收信号的函数，处于阻塞等待状态，再启动notify的函数发出信号 from threading import Thread, Condition '''聊天 Peaple1 : How are you? Peaple2 : I`m fine, thank you! Peaple1 : What`s your job? Peaple2 : My job is teacher. ''' def Peaple1(condition): with condition: print('Peaple1 : ', 'How are you?') condition.notify() condition.wait() print('Peaple1 : ', 'What`s your job?') condition.notify() condition.wait() def Peaple2(condition): with condition: condition.wait() print('Peaple2 : ', 'I`m fine, thank you!') condition.notify() condition.wait() print('Peaple2 : ', 'My job is teacher.') condition.notify() if __name__ == '__main__': cond = Condition() thread1 = Thread(target=Peaple1, args=(cond,)) thread2 = Thread(target=Peaple2, args=(cond,)) # 此处thread2要比thread1提前启动，因为notify必须要有wait接收；如果先启动thread1,没有wait接收notify信号，那么将会死锁。 thread2.start() thread1.start() # thread1.join() # thread2.join() Semphore 该类实现信号量对象。信号量管理一个原子计数器，表示release（）调用的数量减去acquire（）调用的数量加上一个初始值。如果需要，acquire（）方法会阻塞，直到它可以返回而不使计数器为负。如果没有给出，则值默认为1。 #Semaphore 是用于控制进入数量的锁 #文件， 读、写， 写一般只是用于一个线程写，读可以允许有多个 import threading import time class HtmlSpider(threading.Thread): def __init__(self, url, sem): super().__init__() self.url = url self.sem = sem def run(self): time.sleep(2) print("Download {html} success\n".format(html=self.url)) self.sem.release() class UrlProducer(threading.Thread): def __init__(self, sem): super().__init__() self.sem = sem def run(self): for i in range(20): self.sem.acquire() html_thread = HtmlSpider("https://www.baidu.com/{}".format(i), self.sem) html_thread.start() if __name__ == "__main__": # 控制锁的数量, 每次同时会有3个线程获得锁，然后输出 sem = threading.Semaphore(3) url_producer = UrlProducer(sem) url_producer.start()

notice: Using locks, conditions, and semaphores in the with statement All of the objects provided by this module that have acquire() and release() methods can be used as context managers for a with statement. The acquire() method will be called when the block is entered, and release() will be called when the block is exited. Hence, the following snippet:

with some_lock: # do something... is equivalent to: some_lock.acquire() try: # do something... finally: some_lock.release()

Currently, Lock, RLock, Condition, Semaphore, and BoundedSemaphore objects may be used as with statement context managers.

(三)简单介绍多进程编程

多进程编程中进程间不能实现全局变量共享，也不能使用queue.Queue多进程编程通信需要使用Queue，Pipe如果使用进程池进程编程需要使用Manger的实例的queue来实现通信