本篇內容介紹了“Kubernetes PodGC Controller怎么配置”的有關知識,在實際案例的操作過程中,不少人都會遇到這樣的困境,接下來就讓小編帶領大家學習一下如何處理這些情況吧!希望大家仔細閱讀,能夠學有所成!
創(chuàng)新互聯(lián)建站是一家專注于成都做網(wǎng)站、成都網(wǎng)站制作和德陽服務器托管的網(wǎng)絡公司,有著豐富的建站經(jīng)驗和案例。
關于PodGC Controller的相關配置(kube-controller-manager配置),一共只有兩個:
flag | default value | comments |
---|---|---|
--controllers stringSlice | * | 這里配置需要enable的controlllers列表,podgc當然也可以在這里設置是都要enable or disable,默認podgc是在enable列表中的。 |
--terminated-pod-gc-threshold int32 | 12500 | Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled. (default 12500) |
PodGC Controller是在kube-controller-manager Run的時候啟動的。CMServer Run時會invoke StartControllers將預先注冊的enabled Controllers遍歷并逐個啟動。
cmd/kube-controller-manager/app/controllermanager.go:180 func Run(s *options.CMServer) error { ... err := StartControllers(newControllerInitializers(), s, rootClientBuilder, clientBuilder, stop) ... }
在newControllerInitializers注冊了所有一些常規(guī)Controllers
及其對應的start方法,為什么說這些是常規(guī)的Controllers呢,因為還有一部分Controllers沒在這里進行注冊,比如非常重要的service Controller,node Controller等,我把這些稱為非常規(guī)Controllers
。
func newControllerInitializers() map[string]InitFunc { controllers := map[string]InitFunc{} controllers["endpoint"] = startEndpointController ... controllers["podgc"] = startPodGCController ... return controllers }
因此CMServer最終是invoke startPodGCController來啟動PodGC Controller的。
cmd/kube-controller-manager/app/core.go:66 func startPodGCController(ctx ControllerContext) (bool, error) { go podgc.NewPodGC( ctx.ClientBuilder.ClientOrDie("pod-garbage-collector"), ctx.InformerFactory.Core().V1().Pods(), int(ctx.Options.TerminatedPodGCThreshold), ).Run(ctx.Stop) return true, nil }
startPodGCController內容很簡單,啟動一個goruntine協(xié)程,創(chuàng)建PodGC并啟動執(zhí)行。
我們先來看看PodGCController的定義。
pkg/controller/podgc/gc_controller.go:44 type PodGCController struct { kubeClient clientset.Interface podLister corelisters.PodLister podListerSynced cache.InformerSynced deletePod func(namespace, name string) error terminatedPodThreshold int }
kubeClient: 用來跟APIServer通信的client。
PodLister: PodLister helps list Pods.
podListerSynced: 用來判斷PodLister是否Has Synced。
deletePod: 調用apiserver刪除對應pod的接口。
terminatedPodThreshold: 對應--terminated-pod-gc-threshold
的配置,默認為12500。
pkg/controller/podgc/gc_controller.go:54 func NewPodGC(kubeClient clientset.Interface, podInformer coreinformers.PodInformer, terminatedPodThreshold int) *PodGCController { if kubeClient != nil && kubeClient.Core().RESTClient().GetRateLimiter() != nil { metrics.RegisterMetricAndTrackRateLimiterUsage("gc_controller", kubeClient.Core().RESTClient().GetRateLimiter()) } gcc := &PodGCController{ kubeClient: kubeClient, terminatedPodThreshold: terminatedPodThreshold, deletePod: func(namespace, name string) error { glog.Infof("PodGC is force deleting Pod: %v:%v", namespace, name) return kubeClient.Core().Pods(namespace).Delete(name, metav1.NewDeleteOptions(0)) }, } gcc.podLister = podInformer.Lister() gcc.podListerSynced = podInformer.Informer().HasSynced return gcc }
創(chuàng)建PodGC Controller時其實只是把相關的PodGCController元素進行賦值。注意deletePod方法定義時的參數(shù)metav1.NewDeleteOptions(0)
,表示立即刪除pod,沒有grace period。
創(chuàng)建完PodGC Controller后,接下來就是執(zhí)行Run方法啟動執(zhí)行了。
pkg/controller/podgc/gc_controller.go:73 func (gcc *PodGCController) Run(stop <-chan struct{}) { if !cache.WaitForCacheSync(stop, gcc.podListerSynced) { utilruntime.HandleError(fmt.Errorf("timed out waiting for caches to sync")) return } go wait.Until(gcc.gc, gcCheckPeriod, stop) <-stop }
每100ms都會去檢查對應的PodLister是否Has Synced,直到Has Synced。
啟動goruntine協(xié)程,每執(zhí)行完一次gcc.gc進行Pod回收后,等待20s,再次執(zhí)行gcc.gc,直到收到stop信號。
pkg/controller/podgc/gc_controller.go:83 func (gcc *PodGCController) gc() { pods, err := gcc.podLister.List(labels.Everything()) if err != nil { glog.Errorf("Error while listing all Pods: %v", err) return } if gcc.terminatedPodThreshold > 0 { gcc.gcTerminated(pods) } gcc.gcOrphaned(pods) gcc.gcUnscheduledTerminating(pods) }
gcc.gc是最終的pod回收邏輯:
調從PodLister中去除所有的pods(不設置過濾)
如果terminatedPodThreshold
大于0,則調用gcc.gcTerminated(pods)
回收那些超出Threshold的Pods。
調用gcc.gcOrphaned(pods)
回收Orphaned pods。
調用gcc.gcUnscheduledTerminating(pods)
回收UnscheduledTerminating pods。
注意:
gcTerminated和gcOrphaned,gcUnscheduledTerminating這三個gc都是串行執(zhí)行的。
gcTerminated刪除超出閾值的pods的刪除動作是并行的,通過
sync.WaitGroup
等待所有對應的pods刪除完成后,gcTerminated才會結束返回,才能開始后面的gcOrphaned.gcOrphaned,gcUnscheduledTerminatin,gcUnscheduledTerminatin內部都是串行gc pods的。
func (gcc *PodGCController) gcTerminated(pods []*v1.Pod) { terminatedPods := []*v1.Pod{} for _, pod := range pods { if isPodTerminated(pod) { terminatedPods = append(terminatedPods, pod) } } terminatedPodCount := len(terminatedPods) sort.Sort(byCreationTimestamp(terminatedPods)) deleteCount := terminatedPodCount - gcc.terminatedPodThreshold if deleteCount > terminatedPodCount { deleteCount = terminatedPodCount } if deleteCount > 0 { glog.Infof("garbage collecting %v pods", deleteCount) } var wait sync.WaitGroup for i := 0; i < deleteCount; i++ { wait.Add(1) go func(namespace string, name string) { defer wait.Done() if err := gcc.deletePod(namespace, name); err != nil { // ignore not founds defer utilruntime.HandleError(err) } }(terminatedPods[i].Namespace, terminatedPods[i].Name) } wait.Wait() }
遍歷所有pods,過濾出所有Terminated Pods(Pod.Status.Phase不為Pending, Running, Unknow的Pods).
計算terminated pods數(shù)與terminatedPodThreshold的(超出)差值deleteCount。
啟動deleteCount數(shù)量的goruntine協(xié)程,并行調用gcc.deletePod(invoke apiserver's api)方法立刻刪除對應的pod。
// gcOrphaned deletes pods that are bound to nodes that don't exist. func (gcc *PodGCController) gcOrphaned(pods []*v1.Pod) { glog.V(4).Infof("GC'ing orphaned") // We want to get list of Nodes from the etcd, to make sure that it's as fresh as possible. nodes, err := gcc.kubeClient.Core().Nodes().List(metav1.ListOptions{}) if err != nil { return } nodeNames := sets.NewString() for i := range nodes.Items { nodeNames.Insert(nodes.Items[i].Name) } for _, pod := range pods { if pod.Spec.NodeName == "" { continue } if nodeNames.Has(pod.Spec.NodeName) { continue } glog.V(2).Infof("Found orphaned Pod %v assigned to the Node %v. Deleting.", pod.Name, pod.Spec.NodeName) if err := gcc.deletePod(pod.Namespace, pod.Name); err != nil { utilruntime.HandleError(err) } else { glog.V(0).Infof("Forced deletion of orphaned Pod %s succeeded", pod.Name) } } }
gcOrphaned用來刪除那些bind的node已經(jīng)不存在的pods。
調用apiserver接口,獲取所有的Nodes。
遍歷所有pods,如果pod bind的NodeName不為空且不包含在剛剛獲取的所有Nodes中,則串行逐個調用gcc.deletePod刪除對應的pod。
pkg/controller/podgc/gc_controller.go:167 // gcUnscheduledTerminating deletes pods that are terminating and haven't been scheduled to a particular node. func (gcc *PodGCController) gcUnscheduledTerminating(pods []*v1.Pod) { glog.V(4).Infof("GC'ing unscheduled pods which are terminating.") for _, pod := range pods { if pod.DeletionTimestamp == nil || len(pod.Spec.NodeName) > 0 { continue } glog.V(2).Infof("Found unscheduled terminating Pod %v not assigned to any Node. Deleting.", pod.Name) if err := gcc.deletePod(pod.Namespace, pod.Name); err != nil { utilruntime.HandleError(err) } else { glog.V(0).Infof("Forced deletion of unscheduled terminating Pod %s succeeded", pod.Name) } } }
gcUnscheduledTerminating刪除那些terminating并且還沒調度到某個node的pods。
遍歷所有pods,過濾那些terminating(pod.DeletionTimestamp != nil
)并且未調度成功的(pod.Spec.NodeName為空)的pods。
串行逐個調用gcc.deletePod刪除對應的pod。
“Kubernetes PodGC Controller怎么配置”的內容就介紹到這里了,感謝大家的閱讀。如果想了解更多行業(yè)相關的知識可以關注創(chuàng)新互聯(lián)網(wǎng)站,小編將為大家輸出更多高質量的實用文章!
當前標題:KubernetesPodGCController怎么配置
分享路徑:http://vcdvsql.cn/article20/pdehco.html
成都網(wǎng)站建設公司_創(chuàng)新互聯(lián),為您提供小程序開發(fā)、靜態(tài)網(wǎng)站、網(wǎng)站導航、微信公眾號、品牌網(wǎng)站建設、外貿網(wǎng)站建設
聲明:本網(wǎng)站發(fā)布的內容(圖片、視頻和文字)以用戶投稿、用戶轉載內容為主,如果涉及侵權請盡快告知,我們將會在第一時間刪除。文章觀點不代表本網(wǎng)站立場,如需處理請聯(lián)系客服。電話:028-86922220;郵箱:631063699@qq.com。內容未經(jīng)允許不得轉載,或轉載時需注明來源: 創(chuàng)新互聯(lián)