前面文章分析过,通过反馈报文RR和REMB,可以得到对端期望的发送码率,发送端需要根据这个码率来动态调节自己的发送码率,发送码率的控制主要有两个模块,VieEncoder模块和PacedSender模块,本文章描述PacedSender模块如何调节发送码率。
先看类图:
PacedSender类中有PacketQueue类的对象,我们可以简单的理解为一个队列,别的线程将一张图片编码,打包成RTP后,便会将一个Packet插入到PacketQueue队列中,而PacedSender在另一个线程执行Process(), 根据设定的码率来延时一定的时间,然后从PacketQueue队列中取出Packet,再发送到后面的环节。
插入packet的代码为:
void PacedSender::InsertPacket(RtpPacketSender::Priority priority, uint32_t ssrc, uint16_t sequence_number, int64_t capture_time_ms, size_t bytes, bool retransmission) { if (capture_time_ms < 0) capture_time_ms = now_ms; packets_->Push(paced_sender::Packet(priority, ssrc, sequence_number, capture_time_ms, now_ms, bytes, retransmission, packet_counter_++)); }从类图我们可以看到,PacedSender实现了Module接口,在pace线程中会循环调用Process函数。实际上码率控制就是在里面进行的。我们先看下PacedSender更新目标码率的代码:
void PacedSender::UpdateBitrate(int bitrate_kbps, int max_bitrate_kbps, int min_bitrate_kbps) { // Don't set media bitrate here as it may be boosted in order to meet max // queue time constraint. Just update max_bitrate_kbps_ and let media_budget_ // be updated in Process(). padding_budget_->set_target_rate_kbps(min_bitrate_kbps); bitrate_bps_ = 1000 * bitrate_kbps; max_bitrate_kbps_ = max_bitrate_kbps; }我们看到,最小码率min_bitrate_kbps设置给了padding_budget_,注释里还说在Process()会把最大码率max_bitrate_kbps设置给media_budget_, 那么padding_budget_和media_budget_是干嘛用的呢,其实他们就是分别用来控制码率的下限和上限的,我想了很久也不知道怎么描述清楚他们控制的原理,其实还是比较简单的,我先写出Process()的一些注释吧:
void PacedSender::Process() { int64_t now_us = clock_->TimeInMicroseconds(); //当前时间 CriticalSectionScoped cs(critsect_.get()); int64_t elapsed_time_ms = (now_us - time_last_update_us_ + 500) / 1000; //自从上一次process过了多久 time_last_update_us_ = now_us; int target_bitrate_kbps = max_bitrate_kbps_; // TODO(holmer): Remove the !paused_ check when issue 5307 has been fixed. if (!paused_ && elapsed_time_ms > 0) { size_t queue_size_bytes = packets_->SizeInBytes(); if (queue_size_bytes > 0) { // Assuming equal size packets and input/output rate, the average packet // has avg_time_left_ms left to get queue_size_bytes out of the queue, if // time constraint shall be met. Determine bitrate needed for that. //虽然还没有完全明白,但是可以猜测,码率必须有一个下限值,才能让packet队列保持 //出队入队的平衡,不然码率过小,pakcet队列会越来越大 packets_->UpdateQueueTime(clock_->TimeInMilliseconds()); int64_t avg_time_left_ms = std::max<int64_t>( 1, kMaxQueueLengthMs - packets_->AverageQueueTimeMs()); int min_bitrate_needed_kbps = static_cast<int>(queue_size_bytes * 8 / avg_time_left_ms); //目标码率不能低于这个码率 if (min_bitrate_needed_kbps > target_bitrate_kbps) target_bitrate_kbps = min_bitrate_needed_kbps; } media_budget_->set_target_rate_kbps(target_bitrate_kbps); int64_t delta_time_ms = std::min(kMaxIntervalTimeMs, elapsed_time_ms); //发送间隔不能小于30ms UpdateBytesPerInterval(delta_time_ms); //增加media_budget_和padding_budget_的bytes_remaining_ } while (!packets_->Empty()) { if (media_budget_->bytes_remaining() == 0 && !prober_->IsProbing()) //从这里可以看出,必须要在media_budget_的bytes_remaining_大于0时,才能发送packet return; // Since we need to release the lock in order to send, we first pop the // element from the priority queue but keep it in storage, so that we can // reinsert it if send fails. const paced_sender::Packet& packet = packets_->BeginPop(); //从优先堆中取出一个packet if (SendPacket(packet)) { //这里只是从优先堆中取出packet,没有从List中取 // Send succeeded, remove it from the queue. packets_->FinalizePop(packet); //发送成功,才从list中除去packet if (prober_->IsProbing()) return; } else { // Send failed, put it back into the queue. packets_->CancelPop(packet); //如果失败,要把packet放回list return; } } // TODO(holmer): Remove the paused_ check when issue 5307 has been fixed. if (paused_ || !packets_->Empty()) //从这里可以看出,只有packets_为空时,才会有后面的padding_budget_的处理 return; size_t padding_needed; if (prober_->IsProbing()) { padding_needed = prober_->RecommendedPacketSize(); } else { padding_needed = padding_budget_->bytes_remaining(); } if (padding_needed > 0) SendPadding(static_cast<size_t>(padding_needed)); //当队列中已经没有了Packet,为了使码率不低于最小码率,必须填充一些数据发出去 }