-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
1082 lines (759 loc) · 119 KB
/
Copy pathindex.html
File metadata and controls
1082 lines (759 loc) · 119 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Wendi Cai's Blog</title>
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta property="og:type" content="website">
<meta property="og:title" content="Wendi Cai's Blog">
<meta property="og:url" content="http://windyztree.github.io/index.html">
<meta property="og:site_name" content="Wendi Cai's Blog">
<meta property="og:locale" content="en_US">
<meta property="article:author" content="Wendi Cai">
<meta name="twitter:card" content="summary">
<link rel="alternate" href="/atom.xml" title="Wendi Cai's Blog" type="application/atom+xml">
<link rel="shortcut icon" href="/favicon.png">
<link rel="stylesheet" href="/css/style.css">
<link rel="stylesheet" href="/fancybox/jquery.fancybox.min.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/fork-awesome@1.2.0/css/fork-awesome.min.css">
<meta name="generator" content="Hexo 8.1.1"></head>
<body>
<div id="container">
<div id="wrap">
<header id="header">
<div id="banner"></div>
<div id="header-outer" class="outer">
<div id="header-title" class="inner">
<h1 id="logo-wrap">
<a href="/" id="logo">Wendi Cai's Blog</a>
</h1>
</div>
<div id="header-inner" class="inner">
<nav id="main-nav">
<a id="main-nav-toggle" class="nav-icon"><span class="fa fa-bars"></span></a>
<a class="main-nav-link" href="/">Home</a>
<a class="main-nav-link" href="/archives">Archives</a>
</nav>
<nav id="sub-nav">
<a class="nav-icon" href="/atom.xml" title="RSS Feed"><span class="fa fa-rss"></span></a>
<a class="nav-icon nav-search-btn" title="Search"><span class="fa fa-search"></span></a>
</nav>
<div id="search-form-wrap">
<form action="//google.com/search" method="get" accept-charset="UTF-8" class="search-form"><input type="search" name="q" class="search-form-input" placeholder="Search"><button type="submit" class="search-form-submit"></button><input type="hidden" name="sitesearch" value="http://windyztree.github.io"></form>
</div>
</div>
</div>
</header>
<div class="outer">
<section id="main">
<article id="post-openclaw-memory-and-cron" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2026/03/22/openclaw-memory-and-cron/" class="article-date">
<time class="dt-published" datetime="2026-03-21T16:00:00.000Z" itemprop="datePublished">2026-03-22</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/AI/">AI</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2026/03/22/openclaw-memory-and-cron/">小龙虾养成笔记（一）：用 OpenClaw 搭一个有记忆的 AI Agent</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<p>用过 ChatGPT 的人都有过这种体验：每次打开新对话，又得从头介绍一遍”我是谁、我在做什么、我上次跟你说过什么”。对话结束，一切归零。</p>
<p>这不是 AI 不够聪明，而是 LLM 天生无状态——session 结束就失忆。</p>
<p>OpenClaw 解决的就是这个问题。它不是又一个聊天界面，而是一套让 LLM 变成<strong>持久化 Agent</strong> 的框架：有记忆、有闹钟、有手脚，关掉窗口它还在。</p>
<p>和OpenClaw折腾了有段时间了，打算记录一下我的个人使用心得。本文作为第一篇，重点聊两件事：<strong>怎么让它记住东西</strong>，以及<strong>怎么让它自己动起来</strong>。</p>
<hr>
<h2 id="1-架构速览-🏗️"><a href="#1-架构速览-🏗️" class="headerlink" title="1. 架构速览 🏗️"></a>1. 架构速览 🏗️</h2><p>先看全貌。OpenClaw 的核心组件：</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">Gateway（常驻进程）</span><br><span class="line"> ├── Agent（你的 AI 助手）</span><br><span class="line"> │ ├── Workspace（文件系统 = 状态）</span><br><span class="line"> │ │ ├── SOUL.md ← 身份定义</span><br><span class="line"> │ │ ├── USER.md ← 用户画像</span><br><span class="line"> │ │ ├── MEMORY.md ← 长期记忆</span><br><span class="line"> │ │ └── memory/ ← 每日日志</span><br><span class="line"> │ ├── Skills（可插拔能力模块）</span><br><span class="line"> │ └── Cron（定时任务）</span><br><span class="line"> ├── Channel（飞书 / Discord / Telegram）</span><br><span class="line"> └── Session（一次对话）</span><br></pre></td></tr></table></figure>
<p>几个关键设计决策：</p>
<ul>
<li><strong>文件系统即状态</strong>。记忆就是 markdown 文件，不需要向量数据库，<code>cat</code> 就能看，<code>git</code> 就能版本管理。</li>
<li><strong>Gateway 常驻</strong>。不是”用的时候启动”，而是一直在跑。这样 cron 任务和心跳才有意义。</li>
<li><strong>Skill 热插拔</strong>。新能力以 SKILL.md 的形式挂载，不改代码，重启即生效。</li>
</ul>
<hr>
<h2 id="2-持久化记忆：三层架构-🧠"><a href="#2-持久化记忆：三层架构-🧠" class="headerlink" title="2. 持久化记忆：三层架构 🧠"></a>2. 持久化记忆：三层架构 🧠</h2><p>这是我觉得 OpenClaw 最优雅的设计。记忆分三层，各司其职：</p>
<h3 id="第一层：SOUL-md-—-我是谁"><a href="#第一层：SOUL-md-—-我是谁" class="headerlink" title="第一层：SOUL.md — 我是谁"></a>第一层：SOUL.md — 我是谁</h3><figure class="highlight markdown"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="section"># SOUL.md</span></span><br><span class="line"><span class="emphasis">_You're not a chatbot. You're becoming someone._</span></span><br><span class="line"></span><br><span class="line"><span class="section">## Vibe</span></span><br><span class="line">随意、轻松、像朋友聊天。不端着，不啰嗦。用中文交流。</span><br></pre></td></tr></table></figure>
<p>这个文件定义 Agent 的”人格”。每次 session 启动时自动加载，所以它<strong>永远知道自己是谁</strong>。不需要你每次提醒”你是一个轻松风格的助手”。</p>
<h3 id="第二层：MEMORY-md-—-长期记忆"><a href="#第二层：MEMORY-md-—-长期记忆" class="headerlink" title="第二层：MEMORY.md — 长期记忆"></a>第二层：MEMORY.md — 长期记忆</h3><figure class="highlight markdown"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="section"># 长期记忆</span></span><br><span class="line"></span><br><span class="line"><span class="section">## 关于用户</span></span><br><span class="line"><span class="bullet">-</span> 时区 UTC+8</span><br><span class="line"><span class="bullet">-</span> 喜欢简洁的交流风格</span><br><span class="line"></span><br><span class="line"><span class="section">## 环境</span></span><br><span class="line"><span class="bullet">-</span> 运行在 WSL2 (Ubuntu) on Windows</span><br><span class="line"></span><br><span class="line"><span class="section">## 教训</span></span><br><span class="line"><span class="bullet">-</span> 安装 skill 后需要重启 gateway 才能刷新</span><br><span class="line"><span class="bullet">-</span> 不同 agent 发飞书消息要用各自的 accountId</span><br></pre></td></tr></table></figure>
<p>这是你与小龙虾的每一次互动后沉淀下来的精华。不是对话记录的堆砌，而是从日常交互中提炼出来的重要信息。为了确保MEMORY.md能及时更新，我设置了一个每日反思任务，让我的龙虾在每天晚上回顾过去24小时的对话，将关键内容存入MEMORY.md。</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">【每日自我反思】</span><br><span class="line">...</span><br><span class="line">3. 记忆维护：回顾过去24小时对话（sessions_list + sessions_history），关键内容归入 MEMORY.md；清理过时内容</span><br><span class="line">...</span><br></pre></td></tr></table></figure>
<h3 id="第三层：memory-YYYY-MM-DD-md-—-每日日志"><a href="#第三层：memory-YYYY-MM-DD-md-—-每日日志" class="headerlink" title="第三层：memory/YYYY-MM-DD.md — 每日日志"></a>第三层：memory/YYYY-MM-DD.md — 每日日志</h3><figure class="highlight markdown"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="section"># 2026-03-21 (周六)</span></span><br><span class="line"></span><br><span class="line"><span class="section">## 完成的事情</span></span><br><span class="line"><span class="bullet">-</span> 迁移了 quantified<span class="emphasis">_self 数据格式</span></span><br><span class="line"><span class="emphasis">- 封装了 agent-learn skill</span></span><br><span class="line"><span class="emphasis"></span></span><br><span class="line"><span class="emphasis">## 学习的内容</span></span><br><span class="line"><span class="emphasis">- 学了《段永平投资问答录》，笔记在 brain/投资/books/</span></span><br></pre></td></tr></table></figure>
<p>这是小龙虾的日记，记录与你交流的内容的要点总结，按天存储。只有今天和昨天的会被加载到 session context 里，不会把所有历史都塞进去。</p>
<h3 id="Compaction：对话压缩机制"><a href="#Compaction：对话压缩机制" class="headerlink" title="Compaction：对话压缩机制"></a>Compaction：对话压缩机制</h3><p>Session 的上下文窗口是有限的。聊久了怎么办？</p>
<p>OpenClaw 用了 <strong>compaction</strong> 机制。当对话超过一定长度时，系统会把前面的对话压缩成一段摘要，释放 token 空间。但压缩前会触发一个 <strong>pre-compaction flush</strong>——给 Agent 一个机会把重要信息写到文件里。</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">对话进行中...</span><br><span class="line"> ↓ 上下文快满了</span><br><span class="line"> ↓ 触发 pre-compaction flush</span><br><span class="line"> ↓ Agent 把关键信息写入 memory/2026-03-21.md</span><br><span class="line"> ↓ 压缩旧对话为摘要</span><br><span class="line"> ↓ 继续对话</span><br></pre></td></tr></table></figure>
<p>但实测下来，自动compaction可能会导致最近一两回合对话的信息丢失（可能因为这部分对话正好卡在了pre-compaction flush和compaction之间）。如果这部分对话正好包含关键信息的话，龙虾“失忆”的现象就会特别明显。</p>
<p>所以我在 AGENTS.md 里加了一条规则：<strong>做完就记 memory，不等 pre-compaction flush</strong>。因为 compaction 随时可能发生，重要的东西要立刻落盘。</p>
<hr>
<h2 id="3-自主调度：Cron-Heartbeat-⏰"><a href="#3-自主调度：Cron-Heartbeat-⏰" class="headerlink" title="3. 自主调度：Cron + Heartbeat ⏰"></a>3. 自主调度：Cron + Heartbeat ⏰</h2><p>有了记忆，Agent 不会失忆了。但它还是被动的——你不说话它就歇着。</p>
<h3 id="Cron：定时任务"><a href="#Cron：定时任务" class="headerlink" title="Cron：定时任务"></a>Cron：定时任务</h3><p>OpenClaw 内置了 cron 系统。每个 cron 任务在独立的 session 里运行，互不干扰。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 创建一个每天 22:00 执行的反思任务</span></span><br><span class="line">openclaw cron create \</span><br><span class="line"> --name <span class="string">"self-reflect"</span> \</span><br><span class="line"> --schedule <span class="string">"0 22 * * *"</span> \</span><br><span class="line"> --timezone <span class="string">"Asia/Shanghai"</span> \</span><br><span class="line"> --agent main \</span><br><span class="line"> --message <span class="string">"回顾今天的对话，记录教训，更新记忆..."</span></span><br></pre></td></tr></table></figure>
<p>例如，我现在跑着这些 cron：</p>
<table>
<thead>
<tr>
<th>时间</th>
<th>任务</th>
<th>做什么</th>
</tr>
</thead>
<tbody><tr>
<td>19:00</td>
<td>learn</td>
<td>从知识库学习新文章</td>
</tr>
<tr>
<td>22:00</td>
<td>self-reflect</td>
<td>每日反思</td>
</tr>
<tr>
<td>23:00</td>
<td>ai-daily-news</td>
<td>自动生成 AI 日报</td>
</tr>
<tr>
<td>23:20</td>
<td>investment-analysis</td>
<td>对给定标的进行分析并给出投资建议</td>
</tr>
</tbody></table>
<h3 id="Heartbeat：心跳巡查"><a href="#Heartbeat：心跳巡查" class="headerlink" title="Heartbeat：心跳巡查"></a>Heartbeat：心跳巡查</h3><p>Cron 是精确定时，Heartbeat 是模糊轮询。Gateway 每隔一段时间 ping 一下 Agent，Agent 决定要不要做点什么。</p>
<p>适合心跳的场景：批量检查（邮件+日历+通知一起看）、不需要精确时间的巡查、比较轻量级的任务。</p>
<p>适合 cron 的场景：精确时间、需要隔离的 session、需要指定不同 model 的任务。</p>
<p>我目前还没有找到需要Heartbeat的场景，因此我的HEARTBEAT.md 里就一行：</p>
<figure class="highlight markdown"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">无需执行任何操作。</span><br></pre></td></tr></table></figure>
<hr>
<h2 id="4-Skill：可复用的能力封装-🔧"><a href="#4-Skill：可复用的能力封装-🔧" class="headerlink" title="4. Skill：可复用的能力封装 🔧"></a>4. Skill：可复用的能力封装 🔧</h2><p>当你发现多个 Agent 在 cron prompt 里写着差不多的逻辑时，就该封装成 Skill 了。</p>
<h3 id="封装前：每个-cron-写一大段-prompt"><a href="#封装前：每个-cron-写一大段-prompt" class="headerlink" title="封装前：每个 cron 写一大段 prompt"></a>封装前：每个 cron 写一大段 prompt</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">investment-learn 的 prompt（25 行）：</span><br><span class="line">1. 读取 brain/.notion-manifest.json...</span><br><span class="line">2. 读取 data/learned-articles.json...</span><br><span class="line">3. 筛选 tagDir 为 "投资" 的文章...</span><br><span class="line">...</span><br><span class="line"></span><br><span class="line">job-coach-learn 的 prompt（25 行）：</span><br><span class="line">1. 读取 brain/.notion-manifest.json... ← 重复！</span><br><span class="line">2. 读取 data/learned-articles.json... ← 重复！</span><br><span class="line">3. 筛选 tagDir 为 "求职" 的文章...</span><br><span class="line">...</span><br></pre></td></tr></table></figure>
<h3 id="封装后：一个-Skill-参数化调用"><a href="#封装后：一个-Skill-参数化调用" class="headerlink" title="封装后：一个 Skill + 参数化调用"></a>封装后：一个 Skill + 参数化调用</h3><p><strong>SKILL.md</strong>（定义一次）：</p>
<figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">---</span></span><br><span class="line"><span class="attr">name:</span> <span class="string">agent-learn</span></span><br><span class="line"><span class="attr">description:</span> <span class="string">Agent</span> <span class="string">知识学习</span> <span class="string">skill。从多个数据源学习新内容。</span></span><br><span class="line"><span class="meta">---</span></span><br><span class="line"><span class="comment">## 参数</span></span><br><span class="line"><span class="string">|</span> <span class="string">参数</span> <span class="string">|</span> <span class="string">说明</span> <span class="string">|</span></span><br><span class="line"><span class="string">|------|------|</span></span><br><span class="line"><span class="string">|</span> <span class="string">brain_tag</span> <span class="string">|</span> <span class="string">知识库的</span> <span class="string">tagDir</span> <span class="string">名称</span> <span class="string">|</span></span><br><span class="line"><span class="string">|</span> <span class="string">drive_folder_token</span> <span class="string">|</span> <span class="string">飞书云文档文件夹（可选）|</span></span><br></pre></td></tr></table></figure>
<p><strong>cron prompt</strong>（精简到 3 行）：</p>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">按 agent-learn skill 学习新内容。参数：</span><br><span class="line">- brain_tag: 投资</span><br><span class="line">- drive_folder_token: XXX</span><br></pre></td></tr></table></figure>
<p>Skill 的三级加载机制也值得一提：</p>
<ol>
<li><strong>Metadata</strong>（name + description）— 始终在 context 里，~100 词</li>
<li><strong>SKILL.md body</strong> — 匹配到才加载，<5k 词</li>
<li><strong>references/ 和 scripts/</strong> — 按需读取，不限大小</li>
</ol>
<p>这保证了：装 20 个 Skill 不会撑爆 context，只有用到的才会加载。</p>
<hr>
<p>养虾一段时间，最直观的感受是：<strong>它开始像一个”人”了</strong>。不是说它有多聪明，而是更像一个身边的伙伴——知道昨天发生了什么，记得我给过的反馈，懂得我的喜好，具有主观能动性（会自我反思和主动干活）。</p>
<p>当然坑也踩了不少，也没少花时间在给龙虾“治病”上。但这正是学习的过程与折腾的乐趣所在。</p>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2026/03/22/openclaw-memory-and-cron/" data-id="cuidiqmB_foSxodWOA4mnorfk" data-title="小龙虾养成笔记（一）：用 OpenClaw 搭一个有记忆的 AI Agent" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/AI-Agent/" rel="tag">AI Agent</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Heartbeat/" rel="tag">Heartbeat</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Memory/" rel="tag">Memory</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Openclaw/" rel="tag">Openclaw</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Skill/" rel="tag">Skill</a></li></ul>
</footer>
</div>
</article>
<article id="post-cri_dockerd_boot_fix" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2026/01/31/cri_dockerd_boot_fix/" class="article-date">
<time class="dt-published" datetime="2026-01-30T16:00:00.000Z" itemprop="datePublished">2026-01-31</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/Kubernetes/">Kubernetes</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2026/01/31/cri_dockerd_boot_fix/">Kubernetes 故障排查：API Server 无法连接与 CRI-Dockerd 开机自动启动失败</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<h2 id="1-问题现象"><a href="#1-问题现象" class="headerlink" title="1. 问题现象"></a>1. 问题现象</h2><p>在部署 Kubernetes v1.24+ 版本（如 v1.35）并使用 Docker 作为运行时环境时，重启 Master 节点后，执行 <code>kubectl get nodes</code> 出现连接被拒绝的报错：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">E0123 06:25:39.253691 5936 memcache.go:265] <span class="string">"Unhandled Error"</span> err=<span class="string">"couldn't get current server API group list: Get \"https://master:6443/api?timeout=32s\": dial tcp 192.168.109.100:6443: connect: connection refused"</span></span><br></pre></td></tr></table></figure>
<p>进一步排查 <code>kubelet</code> 服务日志 (<code>journalctl -xefu kubelet</code>)，发现关键报错：</p>
<blockquote>
<p><code>Error while dialing: dial unix /var/run/cri-dockerd.sock: connect: no such file or directory</code></p>
</blockquote>
<p>检查 <code>cri-dockerd</code> 服务状态时，发现服务未启动。但其实我是配置了<code>cri-dockerd</code>服务开机自动启动的，所以这里的主要问题是为什么自动启动失败。</p>
<p>进一步查看报错，提示 Socket 无法加载主服务：</p>
<figure class="highlight txt"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">systemd[1]: cri-docker.socket: Socket service cri-docker.service not loaded, refusing.</span><br><span class="line">systemd[1]: Failed to listen on cri-docker.socket.</span><br></pre></td></tr></table></figure>
<h2 id="2-原因分析"><a href="#2-原因分析" class="headerlink" title="2. 原因分析"></a>2. 原因分析</h2><ol>
<li><strong>API Server 无法连接的原因</strong>：Kubernetes v1.24+ 移除了 Dockershim，使用 Docker 必须通过 <code>cri-dockerd</code> 中间件。如果 <code>cri-dockerd</code> 未运行，Kubelet 就无法驱动 Docker，导致 API Server 容器无法启动。</li>
<li><strong><code>cri-dockerd</code> 服务启动失败的原因</strong>：<code>cri-dockerd</code> 通常配置为 Socket Activation 模式。这意味着 <code>cri-dockerd.socket</code> 负责创建 <code>/var/run/cri-dockerd.sock</code> 文件并监听请求，收到请求后才拉起 <code>cri-dockerd.service</code>。如果 Socket 没启动，Kubelet 就找不到 sock 文件。</li>
<li><strong>Socket未启动的原因</strong>：<ul>
<li><strong>命名不一致</strong>：Socket 文件配置了 <code>PartOf=cri-dockerd.service</code>，但文件系统中只有 <code>cri-docker.service</code>（少了个 d），导致 Systemd 找不到依赖服务。</li>
<li><strong>语法错误</strong>：Systemd Unit 文件中<strong>不支持行内注释</strong>（如 <code>Requires=xxx # 注释</code>），这会导致整行解析失败，依赖关系失效。</li>
</ul>
</li>
</ol>
<h2 id="3-解决方案"><a href="#3-解决方案" class="headerlink" title="3. 解决方案"></a>3. 解决方案</h2><h3 id="1-先停止并禁用可能存在的旧服务，避免冲突"><a href="#1-先停止并禁用可能存在的旧服务，避免冲突" class="headerlink" title="1. 先停止并禁用可能存在的旧服务，避免冲突"></a>1. 先停止并禁用可能存在的旧服务，避免冲突</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">systemctl stop cri-docker.socket cri-docker.service</span><br><span class="line">systemctl <span class="built_in">disable</span> cri-docker.socket cri-docker.service</span><br><span class="line"><span class="comment"># 如果文件存在，删除它们</span></span><br><span class="line"><span class="built_in">rm</span> -f /etc/systemd/system/cri-docker.socket</span><br><span class="line"><span class="built_in">rm</span> -f /etc/systemd/system/cri-docker.service</span><br></pre></td></tr></table></figure>
<h3 id="2-规范化服务名称与配置"><a href="#2-规范化服务名称与配置" class="headerlink" title="2.规范化服务名称与配置"></a>2.规范化服务名称与配置</h3><p>为了避免混淆，建议统一将 Service 和 Socket 文件命名为 <code>cri-dockerd</code>（带 d）。</p>
<p><strong>修复 Service 文件</strong> (<code>/etc/systemd/system/cri-dockerd.service</code>)：<br><em>注意：去掉所有行内注释，确保无多余换行。</em></p>
<figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="section">[Unit]</span></span><br><span class="line"><span class="attr">Description</span>=CRI Interface for Docker Application Container Engine</span><br><span class="line"><span class="attr">Documentation</span>=https://docs.mirantis.com</span><br><span class="line"><span class="attr">After</span>=network-<span class="literal">on</span>line.target firewalld.service docker.service</span><br><span class="line"><span class="attr">Wants</span>=network-<span class="literal">on</span>line.target</span><br><span class="line"><span class="attr">Requires</span>=cri-dockerd.socket</span><br><span class="line"> </span><br><span class="line"><span class="section">[Service]</span></span><br><span class="line"><span class="attr">Type</span>=notify</span><br><span class="line"><span class="attr">ExecStart</span>=/usr/local/bin/cri-dockerd --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:<span class="number">3.10</span> --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --container-runtime-endpoint=unix:///var/run/cri-dockerd.sock --docker-endpoint=unix:///var/run/docker.sock --cri-dockerd-root-directory=/var/lib/docker</span><br><span class="line"><span class="attr">ExecReload</span>=/bin/kill -s HUP <span class="variable">$MAINPID</span></span><br><span class="line"><span class="attr">TimeoutSec</span>=<span class="number">0</span></span><br><span class="line"><span class="attr">RestartSec</span>=<span class="number">2</span></span><br><span class="line"><span class="attr">Restart</span>=always</span><br><span class="line"><span class="attr">StartLimitBurst</span>=<span class="number">3</span></span><br><span class="line"><span class="attr">StartLimitInterval</span>=<span class="number">60</span>s</span><br><span class="line"><span class="attr">LimitNOFILE</span>=infinity</span><br><span class="line"><span class="attr">LimitNPROC</span>=infinity</span><br><span class="line"><span class="attr">LimitCORE</span>=infinity</span><br><span class="line"><span class="attr">TasksMax</span>=infinity</span><br><span class="line"><span class="attr">Delegate</span>=<span class="literal">yes</span></span><br><span class="line"><span class="attr">KillMode</span>=process</span><br><span class="line"><span class="section">[Install]</span></span><br><span class="line"><span class="attr">WantedBy</span>=multi-user.target</span><br></pre></td></tr></table></figure>
<p><strong>修复 Socket 文件</strong> (<code>/etc/systemd/system/cri-dockerd.socket</code>)：<br><em>注意：PartOf 必须严格匹配 Service 的文件名。</em></p>
<figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="section">[Unit]</span></span><br><span class="line"><span class="attr">Description</span>=CRI Docker Socket for the API</span><br><span class="line"><span class="attr">PartOf</span>=cri-dockerd.service</span><br><span class="line"></span><br><span class="line"><span class="section">[Socket]</span></span><br><span class="line"><span class="attr">ListenStream</span>=/var/run/cri-dockerd.sock</span><br><span class="line"><span class="attr">SocketMode</span>=<span class="number">0660</span></span><br><span class="line"><span class="attr">SocketUser</span>=root</span><br><span class="line"><span class="attr">SocketGroup</span>=docker</span><br><span class="line"></span><br><span class="line"><span class="section">[Install]</span></span><br><span class="line"><span class="attr">WantedBy</span>=sockets.target</span><br></pre></td></tr></table></figure>
<h3 id="3-重新加载并启动"><a href="#3-重新加载并启动" class="headerlink" title="3. 重新加载并启动"></a>3. 重新加载并启动</h3><p>如果之前存在不一致的文件（如 <code>cri-docker.service</code>），需要清理掉，避免 Systemd 冲突。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 重新加载配置</span></span><br><span class="line">systemctl daemon-reload</span><br><span class="line"></span><br><span class="line"><span class="comment"># 启动顺序：先 Socket 后 Service</span></span><br><span class="line">systemctl <span class="built_in">enable</span> --now cri-dockerd.socket</span><br><span class="line">systemctl <span class="built_in">enable</span> --now cri-dockerd.service</span><br></pre></td></tr></table></figure>
<h3 id="4-验证状态"><a href="#4-验证状态" class="headerlink" title="4. 验证状态"></a>4. 验证状态</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">systemctl status cri-dockerd.socket</span><br></pre></td></tr></table></figure>
<p>如果这个是 Active (running)，那么 /var/run/cri-dockerd.sock 就会存在，kubelet也就能正常连接了。</p>
<h3 id="5-最后重启Kubelet"><a href="#5-最后重启Kubelet" class="headerlink" title="5. 最后重启Kubelet"></a>5. 最后重启Kubelet</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">systemctl restart kubelet</span><br></pre></td></tr></table></figure>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2026/01/31/cri_dockerd_boot_fix/" data-id="cuid7qFMJ3U2LY6Pb1Irr5vaU" data-title="Kubernetes 故障排查：API Server 无法连接与 CRI-Dockerd 开机自动启动失败" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/CRI-Dockerd/" rel="tag">CRI-Dockerd</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Kubernetes/" rel="tag">Kubernetes</a></li></ul>
</footer>
</div>
</article>
<article id="post-k8s_deploy_tips" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2026/01/24/k8s_deploy_tips/" class="article-date">
<time class="dt-published" datetime="2026-01-23T16:00:00.000Z" itemprop="datePublished">2026-01-24</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/Kubernetes/">Kubernetes</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2026/01/24/k8s_deploy_tips/">参考掘金 K8s 部署教程时的防踩坑经验分享</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<p>最近在学习 Kubernetes，在掘金上看到一篇部署教程：<a target="_blank" rel="noopener" href="https://juejin.cn/post/7473838374366347303">Kubernetes v1.35 部署实战</a>。</p>
<p>这篇文章讲解得非常清晰，实操性也很强，推荐给想学习搭建 K8s 集群的同学们。</p>
<p>不过，在跟随教程实操的过程中，我遇到了一些配置上的细节问题。为了帮助大家更顺畅地完成部署，我总结了这份经验分享，作为原教程的补充。</p>
<p>注：如果已经遇到了cri-dockerd相关服务开机启动失败的问题，可参考 <a href="https://windyztree.github.io/2026/01/31/cri_dockerd_boot_fix/">Kubernetes 故障排查：API Server 无法连接与 CRI-Dockerd 开机自动启动失败</a> 解决。</p>
<hr>
<h2 id="1-Systemd-配置文件细节"><a href="#1-Systemd-配置文件细节" class="headerlink" title="1. Systemd 配置文件细节"></a>1. Systemd 配置文件细节</h2><p>原文提供了详细的 <code>cri-dockerd</code> 服务配置模板。但在直接使用时，需要注意一些细节。</p>
<h3 id="1-1-避免行内注释"><a href="#1-1-避免行内注释" class="headerlink" title="1.1 避免行内注释"></a>1.1 避免行内注释</h3><p><strong>问题</strong>：<br><code>/etc/systemd/system/cri-dockerd.service</code> 和 <code>/etc/systemd/system/cri-docker.socket</code> 配置文件中如果包含类似 <code>Requires=cri-docker.socket # 依赖socket</code> 这样的行内注释，Systemd 可能会解析失败，报错 <code>Failed to add dependency on #...</code>。</p>
<figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># /etc/systemd/system/cri-dockerd.service：</span></span><br><span class="line"><span class="attr">Requires</span>=cri-docker.socket <span class="comment"># 以前的文件名注释</span></span><br></pre></td></tr></table></figure>
<figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># /etc/systemd/system/cri-docker.socket：</span></span><br><span class="line"><span class="attr">PartOf</span>=cri-docker.service <span class="comment">#systemd cri-docker.servics 文件名</span></span><br></pre></td></tr></table></figure>
<p><strong>解决</strong>：<br>Systemd 的 Unit 文件通常不支持在参数值后直接加 <code>#</code> 进行注释。建议直接删除注释。</p>
<h3 id="1-2-注意长命令换行"><a href="#1-2-注意长命令换行" class="headerlink" title="1.2 注意长命令换行"></a>1.2 注意长命令换行</h3><p><code>/etc/systemd/system/cri-dockerd.service</code> 中的 <code>ExecStart</code> 命令较长，在 <code>pause:3.10</code> 后有一个多余的换行。在复制时，需要删除这个换行，否则会导致命令截断。</p>
<hr>
<h2 id="2-文件名一致性问题"><a href="#2-文件名一致性问题" class="headerlink" title="2. 文件名一致性问题"></a>2. 文件名一致性问题</h2><p>在配置过程中，容易混淆 <code>cri-docker</code> 和 <code>cri-dockerd</code>（是否带 <code>d</code>）。</p>
<p><strong>问题</strong>：<br>如果 Socket 文件中配置了 <code>PartOf=cri-dockerd.service</code>，但实际创建的服务文件名为 <code>cri-docker.service</code>，会导致 Socket 启动后无法正确拉起主服务，Kubelet 也会报错找不到文件。</p>
<p><strong>建议</strong>：<br>保持 Socket 文件和 Service 文件命名的一致性。</p>
<p>我采用的统一命名为：</p>
<ul>
<li><code>/etc/systemd/system/cri-dockerd.service</code></li>
<li><code>/etc/systemd/system/cri-dockerd.socket</code></li>
</ul>
<p>并在 <code>cri-dockerd.service</code> 文件中指定：</p>
<figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="section">[Unit]</span></span><br><span class="line"><span class="attr">Requires</span>=cri-dockerd.socket</span><br></pre></td></tr></table></figure>
<p>在 <code>cri-dockerd.socket</code> 文件中指定：</p>
<figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="section">[Unit]</span></span><br><span class="line"><span class="attr">PartOf</span>=cri-dockerd.service</span><br></pre></td></tr></table></figure>
<hr>
<h2 id="3-kubeadm-初始化配置"><a href="#3-kubeadm-初始化配置" class="headerlink" title="3. kubeadm 初始化配置"></a>3. kubeadm 初始化配置</h2><p>在原教程 <strong>4.6.2 执行kubeadm init命令</strong> 章节中，涉及到 <code>--control-plane-endpoint</code> 字段的配置。</p>
<p><strong>问题</strong>：<br>如果照抄配置文件中的 <code>--control-plane-endpoint=k8smaster</code> 而没有修改，或者将其设置为一个无法解析的域名，<code>kubeadm init</code> 将会失败，提示无法连接到 API Server。</p>
<p><strong>建议</strong>：<br><code>controlPlaneEndpoint</code> 必须是一个<strong>可解析、可访问的地址</strong>，指向 Master 节点的 IP 或域名。</p>
<ul>
<li><strong>推荐做法</strong>：将其设置为 Master 节点的主机名（例如 <code>master</code>）或直接使用 Master 节点的 IP 地址。</li>
<li><strong>前提条件</strong>：如果使用主机名，必须确保该主机名可被解析（如在 <code>/etc/hosts</code> 文件中已经配置了正确的域名解析： <code>192.168.109.100 master</code>）。</li>
</ul>
<hr>
<h2 id="4-小结"><a href="#4-小结" class="headerlink" title="4. 小结"></a>4. 小结</h2><p>以上就是我在使用该教程部署时遇到的一些细节问题。原教程提供了很好的整体步骤，希望这些配置细节可以帮大家避免不必要的排查时间。</p>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2026/01/24/k8s_deploy_tips/" data-id="cuidVENMeNaoHiBYnxmx6EjRk" data-title="参考掘金 K8s 部署教程时的防踩坑经验分享" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Kubernetes/" rel="tag">Kubernetes</a></li></ul>
</footer>
</div>
</article>
<article id="post-sco_scom_ip" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2020/05/10/sco_scom_ip/" class="article-date">
<time class="dt-published" datetime="2020-05-09T16:00:00.000Z" itemprop="datePublished">2020-05-10</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/System-Center-Orchestrator/">System Center Orchestrator</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2020/05/10/sco_scom_ip/">SCORCH - SCOM IP – "Failed to connect. Please verify your connection settings."</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<h3 id="Issue-Definition"><a href="#Issue-Definition" class="headerlink" title="Issue Definition"></a>Issue Definition</h3><p>After regsitering and deploying SCOM Integration Pack (IP) in Orchestrator, we need to configure a connection to specify link to your SCOM management server (Refer to <a target="_blank" rel="noopener" href="https://docs.microsoft.com/en-us/system-center/orchestrator/integration-pack-for-operations-manager?view=sc-orch-2019" title="Configure the connection">Configure the connection</a>).</p>
<p>However, when you click on the <strong>Test Connection</strong> button, you might receive the error <font color=#ff7f50><em>“Failed to connect. Pleae verify your connection settings.”</em></font></p>
<h3 id="Troubleshooting-Tips"><a href="#Troubleshooting-Tips" class="headerlink" title="Troubleshooting Tips"></a>Troubleshooting Tips</h3><h4 id="Verify-Configurations"><a href="#Verify-Configurations" class="headerlink" title="Verify Configurations"></a>Verify Configurations</h4><ul>
<li><p>Ensure the account we use for connection a member of <strong>local Administrator</strong> on the SCOM server and a member of <strong>Operations Manager Administrators</strong> user role.</p>
</li>
<li><p>(<strong>Only for SCOM 2012 IP</strong>) A SCOM console of the same version must be installed on the server that hosts the Runbook Designer. And we should be able to connect to SCOM server from the console with the same account.</p>
<p> Note: SCOM 2016 IP no longer requires SCOM console to be installed on Runbook Designer.</p>
</li>
<li><p>Check whether TLS 1.0 is disabled on SCOM management server and SCORCH server.</p>
<p> We have a known issue that connection fails with the same error in SCORCH 2016 + SCOM 2016 environment with TLS 1.0 disabled. If TLS 1.0 is disabled in your environment, try enabling it an testing the connection again.</p>
</li>
<li><p>Ensure .Net Framwork 3.5 is installed.</p>
</li>
</ul>
<p><a target="_blank" rel="noopener" href="https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/enable-net-framework-35-by-using-windows-powershell" title="Installing .Net Framework 3.5 with PowerShell">Installing .Net Framework 3.5 with PowerShell</a></p>
<p><a target="_blank" rel="noopener" href="https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/enable-net-framework-35-by-using-the-add-roles-and-features-wizard" title="Enable .NET Framework 3.5 by using the Add Roles and Features Wizard">Enable .NET Framework 3.5 by using the Add Roles and Features Wizard</a></p>
<h4 id="Network-Trace"><a href="#Network-Trace" class="headerlink" title="Network Trace"></a>Network Trace</h4><p>If all configurations above are verified, we can use <a target="_blank" rel="noopener" href="https://www.microsoft.com/en-ie/download/details.aspx?id=4865" title="Network Monitor">Network Monitor</a> to capture a network trace and see what causes the failure.</p>
<p>Pay attention to Kerberos records. If there is Kerberos error, check if SCOM SPN is correct: <a target="_blank" rel="noopener" href="https://kevinholman.com/2011/08/08/opsmgr-2012-what-should-the-spns-look-like/" title="OpsMgr 2012: What should the SPN’s look like?">OpsMgr 2012: What should the SPN’s look like?</a></p>
<p>Here is an example of Kerberos error in network trace. The error is <strong>KererosV5:KRB_ERROR -KDC_ERR_S_PRINCIPAL_UNKNOWN(7)</strong> and indicates an SPN issue.</p>
<p><img src="/../images/sco10.jpg"></p>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2020/05/10/sco_scom_ip/" data-id="cuidFluFxcEPyaPzXnxoGiAsP" data-title="SCORCH - SCOM IP – "Failed to connect. Please verify your connection settings."" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Connection/" rel="tag">Connection</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Integration-Pack/" rel="tag">Integration Pack</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Network/" rel="tag">Network</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCOM/" rel="tag">SCOM</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCORCH/" rel="tag">SCORCH</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SPN/" rel="tag">SPN</a></li></ul>
</footer>
</div>
</article>
<article id="post-scom_sql_discovery" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2020/04/29/scom_sql_discovery/" class="article-date">
<time class="dt-published" datetime="2020-04-28T16:00:00.000Z" itemprop="datePublished">2020-04-29</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/System-Center-Operations-Manager/">System Center Operations Manager</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2020/04/29/scom_sql_discovery/">SCOM - SQL DB Engine Discovery Mechanism on Cluster</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<h3 id="In-summary"><a href="#In-summary" class="headerlink" title="In summary"></a>In summary</h3><ul>
<li><p>For any SQL DB Engine on cluster to be discovered, firstly ensure the <strong>SQL Server Network Name</strong> is already discovered in “Agentless Managed”.</p>
</li>
<li><p>If not, check if there is any <strong>orphaned resource</strong> in Windows Cluster.</p>
</li>
</ul>
<h3 id="Details"><a href="#Details" class="headerlink" title="Details"></a>Details</h3><p>For SQL DB Engine on Cluster, the discovery is expected to run on 3 objects.</p>
<ul>
<li>SQL Server Network Name (You can find the name in <strong>Failover Cluster Manager</strong> -> <em>Cluster Name</em> -> <strong>Roles</strong> -> <strong>Server Name</strong>)</li>
<li>Cluster Name (The cluster name in <strong>Failover Cluster Manager</strong>)</li>
<li>Server Name of the Node</li>
</ul>
<p><img src="/../images/scom7.jpg"></p>
<p>Only the execution on <strong>SQL Server Network Name</strong> returns instances on it. Execution on the other 2 objects will show successful, but won’t return any discovered instance. That’s expected.</p>
<p><strong>In SCOM ETL Trace:</strong></p>
<p>Execution on 2012CL (SQL Server Network Name):</p>
<blockquote>
<p>[1]24232.8364::12/26/2019-18:52:44.977 [ExecutionManager] [] [Information] :CTaskTracker::TaskCompleted{tasktracker_cpp1514}Notifying creator of task completion for task 6446, completion reason 0x0, task result is SUCCESS, task output is <DataItem type="System.DiscoveryData" time="2019-12-27T02:52:45.3080007Z" sourceHealthServiceId="d372b2e3-41af-7466-fcfa-e5e97c716492"><DiscoveryType>0</DiscoveryType><DiscoverySourceType>0</DiscoverySourceType><DiscoverySourceObjectId>{bb187d3c-1218-6f43-b6f6-0a0695b96bd7}</DiscoverySourceObjectId><font color=#ff7f50><strong><DiscoverySourceManagedEntity>{e12d4b62-8f91-0b5e-e583-47789cae70dc}</DiscoverySourceManagedEntity></strong></font><br><font color=#009900><strong><ClassInstances></strong></font><br><ClassInstance TypeId="{737b3f9e-b13b-f42a-a7b8-47c625b3cbf0}"><Settings><Setting><Name>MachineName</Name><Value>2012CL.sqlrepro.edu</Value></Setting><Setting><Name>InstanceName</Name><Value>SQL2012AG</Value></Setting><Setting><Name>DisplayName</Name><Value>Resource Pool Group</Value></Setting></Settings></ClassInstance><br><ClassInstance TypeId="{258f9e75-fb47-c6fc-c45b-75df9ca3a53e}"><Settings><Setting><Name>PrincipalName</Name><Value>2012CL.sqlrepro.edu</Value></Setting><Setting><Name>MachineName</Name><Value>2012CL.sqlrepro.edu</Value></Setting><Setting><Name>InstanceName</Name><Value>SQL2012AG</Value></Setting></Settings></ClassInstance><br><ClassInstance TypeId="{01c7eaa4-aaa0-28a2-55ec-af08fb1b7f9d}"><Settings><Setting><Name>MachineName</Name><Value>2012CL.sqlrepro.edu</Value></Setting><Setting><Name>NetbiosComputerName</Name><Value>2012CL</Value></Setting><Setting><Name>NetbiosDomainName</Name><Value>SQLREPRO</Value></Setting><Setting><Name>InstanceName</Name><Value>SQL2012AG</Value></Setting><Setting><Name>DisplayName</Name><Value>2012CL\SQL2012AG</Value></Setting><Setting><Name>ConnectionString</Name><Value>2012CL.sqlrepro.edu\SQL2012AG</Value></Setting><Setting><Name>Edition</Name><Value>Enterprise Edition</Value></Setting><Setting><Name>InstanceID</Name><Value>MSSQL11.SQL2012AG</Value></Setting><Setting><Name>Language</Name><Value>1033</Value></Setting><Setting><Name>Version</Name><Value>11.0.2100.60</Value></Setting><Setting><Name>ServiceName</Name><Value>MSSQL$SQL2012AG</Value></Setting><Setting><Name>ServiceClusterName</Name><Value>SQL SERVER (SQL2012AG)</Value></Setting><Setting><Name>Cluster</Name><Value>True</Value></Setting><Setting><Name>PerformanceCounterObject</Name><Value>MSSQL$SQL2012AG</Value></Setting><Setting><Name>AuthenticationMode</Name><Value>Windows Authentication Mode</Value></Setting><Setting><Name>FullTextSearchServiceName</Name><Value>MSSQLFDLauncher$SQL2012AG</Value></Setting><Setting><Name>FullTextSearchServiceClusterName</Name><Value>SQL Full-text Filter Daemon Launcher (SQL2012AG)</Value></Setting><Setting><Name>AgentName</Name><Value>SQLAgent$SQL2012AG</Value></Setting><Setting><Name>Type</Name><Value>DB Engine</Value></Setting><Setting><Name>AgentClusterName</Name><Value>SQL SERVER AGENT (SQL2012AG)</Value></Setting><Setting><Name>MasterDatabaseLocation</Name><Value>T:\MSSQL11.SQL2012AG\MSSQL\DATA\master.mdf</Value></Setting><Setting><Name>MasterDatabaseLogLocation</Name><Value>T:\MSSQL11.SQL2012AG\MSSQL\DATA\mastlog.ldf</Value></Setting><Setting><Name>ErrorLogLocation</Name><Value>T:\MSSQL11.SQL2012AG\MSSQL\Log\ERRORLOG</Value></Setting><Setting><Name>ServicePackVersion</Name><Value>0</Value></Setting><Setting><Name>AuditLevel</Name><Value>Failure</Value></Setting><Setting><Name>InstallPath</Name><Value>C:\Program Files\Microsoft SQL Server\MSSQL11.SQL2012AG\MSSQL</Value></Setting><Setting><Name>ToolsPath</Name><Value>C:\Program Files\Microsoft SQL Server\110\Tools</Value></Setting><Setting><Name>EnableErrorReporting</Name><Value>False</Value></Setting><Setting><Name>Account</Name><Value>sqlrepro\sqlsvc</Value></Setting><Setting><Name>PrincipalName</Name><Value>2012CL.sqlrepro.edu</Value></Setting><Setting><Name>MonitoringType</Name><Value>Local</Value></Setting></Settings></ClassInstance><br><font color=#009900><strong></ClassInstances></strong></font><br><font color=#009900><strong><RelationshipInstances></strong></font><br><RelationshipInstance TypeId="{5446657d-ac22-0a78-dfb2-36150b58d67d}" SourceTypeId="{c247cbe5-34b2-abf4-8d68-884d4a47cf8b}" TargetTypeId="{01c7eaa4-aaa0-28a2-55ec-af08fb1b7f9d}"><Settings /><SourceRole><Settings /></SourceRole><TargetRole><Settings><Setting><Name>MachineName</Name><Value>2012CL.sqlrepro.edu</Value></Setting><Setting><Name>InstanceName</Name><Value>SQL2012AG</Value></Setting></Settings></TargetRole></RelationshipInstance><br><RelationshipInstance TypeId="{f8c488fb-9ff6-5148-29cb-5b3a9cafce01}" SourceTypeId="{f3a9b2cb-ccdb-3896-045c-fc8749039e7f}" TargetTypeId="{ea99500d-8d52-fc52-b5a5-10dcd1e9d2bd}"><Settings /><SourceRole><Settings /></SourceRole><TargetRole><Settings><Setting><Name>PrincipalName</Name><Value>2012CL.sqlrepro.edu</Value></Setting></Settings></TargetRole></RelationshipInstance><br><RelationshipInstance TypeId="{1e8df0aa-fc37-ace6-cf38-333c67ef4f50}" SourceTypeId="{e817d034-02e8-294c-3509-01ca25481689}" TargetTypeId="{01c7eaa4-aaa0-28a2-55ec-af08fb1b7f9d}"><Settings /><SourceRole><Settings><Setting><Name>PrincipalName</Name><Value>2012CL.sqlrepro.edu</Value></Setting></Settings></SourceRole><TargetRole><Settings><Setting><Name>MachineName</Name><Value>2012CL.sqlrepro.edu</Value></Setting><Setting><Name>InstanceName</Name><Value>SQL2012AG</Value></Setting></Settings></TargetRole></RelationshipInstance><br><font color=#009900><strong></RelationshipInstances></strong></font><br></DataItem>.</p>
</blockquote>
<p>Execution on Cluster2016 (Cluster Name):</p>
<blockquote>
<p>[3]24232.8364::12/26/2019-18:53:05.985 [ExecutionManager] [] [Information] :CTaskTracker::TaskCompleted{tasktracker_cpp1514}Notifying creator of task completion for task 6448, completion reason 0x0, task result is SUCCESS, task output is <DataItem type="System.DiscoveryData" time="2019-12-27T02:53:06.3158795Z" sourceHealthServiceId="d372b2e3-41af-7466-fcfa-e5e97c716492"><DiscoveryType>0</DiscoveryType><DiscoverySourceType>0</DiscoverySourceType><DiscoverySourceObjectId>{bb187d3c-1218-6f43-b6f6-0a0695b96bd7}</DiscoverySourceObjectId><font color=#ff7f50><strong><DiscoverySourceManagedEntity>{124369dd-8822-2f6b-9ce7-9a1b06551b24}</DiscoverySourceManagedEntity></strong></font><br><font color=#009900><strong><ClassInstances /></strong> # No SQL instance is returned here<br><strong><RelationshipInstances /></strong></font><br></DataItem>.</p>
</blockquote>
<p>Execution on Node4 (Server Name of the Node):</p>
<blockquote>
<p>[1]24232.10072::12/26/2019-18:51:54.710 [ExecutionManager] [] [Information] :CTaskTracker::TaskCompleted{tasktracker_cpp1514}Notifying creator of task completion for task 6441, completion reason 0x0, task result is SUCCESS, task output is <DataItem type="System.DiscoveryData" time="2019-12-27T02:51:55.0411614Z" sourceHealthServiceId="d372b2e3-41af-7466-fcfa-e5e97c716492"><DiscoveryType>0</DiscoveryType><DiscoverySourceType>0</DiscoverySourceType><DiscoverySourceObjectId>{bb187d3c-1218-6f43-b6f6-0a0695b96bd7}</DiscoverySourceObjectId><font color=#ff7f50><strong><DiscoverySourceManagedEntity>{264b2a65-07f0-8cce-8757-b309a2bc20b8}</DiscoverySourceManagedEntity></strong></font><br><font color=#009900><strong><ClassInstances /></strong> # No SQL instance is returned here<br><strong><RelationshipInstances /></strong></font><br></DataItem>.</p>
</blockquote>
<p>The 3 GUIDs in the trace are the BME IDs of the 3 objects:<br><img src="/../images/scom8.jpg"></p>
<p>If you check SCOM trace and find the discovery didn’t run on SQL Server Network Name, the first thing is to check whether the discovery target is already discovered in <strong>SCOM Console</strong> -> <strong>Monitoring</strong> -> <strong>Discovered Inventory</strong> -> <em><Target class of the discovery></em>. An object with the <strong>SQL Server Network Name</strong> is probably missing there.</p>
<ul>
<li>Tracing the discovery chain to the source, if the SQL Server Network Name is already discovered and shows up in Agentless Managed, we need to troubleshoot the SQL discovery itself.</li>
<li>Otherwise, if the SQL Server Network Name doesn’t show up in Agentless Managed, the issue is actually a Windows Cluster discovery issue.</li>
</ul>
<p>For the Windows Cluster discovery issue, I followed below steps to resolve it.</p>
<ol>
<li><p><strong>[VERY IMPORTANT]</strong> Confirm agent proxy is enabled on the agents.</p>
</li>
<li><p>Check if the cluster has any orphaned resource blocking the cluster discovery.</p>
<p> a. Run this PowerShell command to check if there is any offline resources that had been removed from cluster manager.</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">get-clusterresource</span> | <span class="built_in">where</span> {<span class="variable">$_</span>.state <span class="operator">-eq</span> <span class="string">'offline'</span>}</span><br></pre></td></tr></table></figure>
<p> b. Run this PowerShell command. Compare the result with the resources in Failover Cluster Manager to find out if there is any orphaned resource.</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">get-clusterresource</span> | <span class="built_in">sort-object</span> <span class="literal">-Property</span> ResourceType</span><br></pre></td></tr></table></figure>
<p> All the resources returned by the command should be found in below areas in Failover Cluster Manager. Otherwise, there is an orphaned resource.</p>
<p> <img src="/../images/scom9.jpg"></p>
<p> <img src="/../images/scom10.jpg"></p>
<p> If any orphaned resource is found, use this command to remove the resource.</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">Remove-ClusterResource</span> <span class="literal">-Name</span> <span class="string">"<Name of the resource>"</span> <span class="literal">-Force</span></span><br></pre></td></tr></table></figure>
<p> After that, flush health service cache on the node and monitor for some time. If it goes well, the SQL Server Network Name and the Cluster Name Object should occur in <strong>Agentless Managed</strong>, then the SQL DB engines should be discovered gradually.</p>
</li>
</ol>
<h3 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h3><p><a target="_blank" rel="noopener" href="https://blog.coeo.com/karlgrambow/2014/10/20/scom-wont-discover-my-sql-server-or-cluster" title="SCOM Won't Discover My SQL Server (Or Cluster) ">SCOM Won’t Discover My SQL Server (Or Cluster) </a></p>
<p><a target="_blank" rel="noopener" href="https://social.technet.microsoft.com/Forums/en-US/5d60cc62-d5d9-4077-b051-092cfdedbe43/cluster-not-appearing-in-agentless-managed?forum=operationsmanagergeneral" title="Cluster not appearing in Agentless Managed">Cluster not appearing in Agentless Managed</a></p>
<p><a target="_blank" rel="noopener" href="https://blog.tyang.org/2014/03/29/discovery-ms-clusters-kind/" title="Discovery for MS Clusters of Any Kind">Discovery for MS Clusters of Any Kind</a></p>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2020/04/29/scom_sql_discovery/" data-id="cuidMXVwZB-KuOitBJv5jGJtz" data-title="SCOM - SQL DB Engine Discovery Mechanism on Cluster" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Cluster/" rel="tag">Cluster</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCOM/" rel="tag">SCOM</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SQL-Discovery/" rel="tag">SQL Discovery</a></li></ul>
</footer>
</div>
</article>
<article id="post-sco_web_fail" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2019/11/24/sco_web_fail/" class="article-date">
<time class="dt-published" datetime="2019-11-23T16:00:00.000Z" itemprop="datePublished">2019-11-24</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/System-Center-Orchestrator/">System Center Orchestrator</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2019/11/24/sco_web_fail/">Fail to connect to Orchestrator Web Service</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<h3 id="Error-Message"><a href="#Error-Message" class="headerlink" title="Error Message"></a>Error Message</h3><p>The same error may occur after applying SCORCH 2016 UR4 and later versions. See <a target="_blank" rel="noopener" href="http://www.wendicai.net/?p=42">Tips and Tricks for Applying SCORCH 2016 UR4 and Later Versions</a>.</p>
<blockquote>
<p>Request Error</p>
<p>The server encountered an error processing the request. The exception message is <font color=#ff7f50><strong>‘An error occurred while executing the command definition. See the inner exception for details.’</strong></font>. See Server logs for more details. The exception stack trace is:</p>
<p>at System.Data.EntityClient.EntityCommandDefinition.ExecuteStoreCommands(EntityCommand entityCommand, CommandBehavior behavior)<br>at System.Data.EntityClient.EntityCommandDefinition.Execute(EntityCommand entityCommand, CommandBehavior behavior)<br>at System.Data.EntityClient.EntityCommand.ExecuteReader(CommandBehavior behavior)<br>at System.Da.EntityCIient.EntityCommand.ExecuteScalar[T_Result](Func`2 resultSelector)<br>at System.Data.Objects.ObjectContext.ExecuteFunction(String functionName, ObjectParameter[] parameters)<br>at Microsoft.SystemCenter.Orchestrator.WebService.OrchestratorContext.ComputeAuthorizationCache(NuIlable`1 tokenld)<br>at Microsoft.SystemCenter.Orchestrator.WebService.OrchestratorContext.OnContextCreated()<br>at invoke_constructor()<br>at System.Data.Services.DataService`1.CreateProvider()<br>at System.Data.Services.DataService`1.HandleRequest()<br>at System.Data.Services.DataService`1.ProcessRequestForMessage(Stream messageBody)<br>at SynclnvokeProcessRequestForMessage(Object, Object[], Object[])<br>at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)<br>at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpt)<br>at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)<br>at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)<br>at System.ServiceModel.Dispatcher.MessageRpC.Process(Boolean isOperationContextSet)at System.ServiceModel.Dispatcher.MessageRpC.Process(Boolean isOperationContextSet)</p>
</blockquote>
<h3 id="Troubleshooting"><a href="#Troubleshooting" class="headerlink" title="Troubleshooting"></a>Troubleshooting</h3><p>This error usually indicates some issues in the Orchestrator DB.</p>
<ol>
<li>Ensure Orchestrator DB disk space is enough.</li>
<li>Run below queries on Orchestrator DB to ensure enough permission, and try again.</li>
</ol>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">GRANT</span> <span class="keyword">EXECUTE</span> <span class="keyword">ON</span> object::[Microsoft.SystemCenter.Orchestrator].[GetSecurityToken] <span class="keyword">TO</span> [Microsoft.SystemCenter.Orchestrator.Operators]</span><br><span class="line"><span class="keyword">GRANT</span> <span class="keyword">SELECT</span> <span class="keyword">ON</span> object::[Microsoft.SystemCenter.Orchestrator.Internal].[Settings] <span class="keyword">TO</span> [Microsoft.SystemCenter.Orchestrator.Operators]</span><br></pre></td></tr></table></figure>
<ol start="3">
<li>Check whether the service <strong><a target="_blank" rel="noopener" href="http://schemas.microsoft.com/SystemCenter/Orchestrator/Maintenance/MaintenanceService">http://schemas.microsoft.com/SystemCenter/Orchestrator/Maintenance/MaintenanceService</a></strong> and queue <strong>[Microsoft.SystemCenter.Orchestrator.Maintenance].[MaintenanceServiceQueue]</strong> are there in Orchestrator DB. If they are missing, we need to create them manually.</li>
</ol>
<p><img src="/../images/sco9.png"></p>
<ol start="4">
<li>If the issue only occurred recently and we have a backup of the Orchestrator DB prior to this issue, we can restore the backup as a quick solution.</li>
</ol>
<h4 id="Log-Analysis"><a href="#Log-Analysis" class="headerlink" title="Log Analysis"></a>Log Analysis</h4><p>Use <a target="_blank" rel="noopener" href="https://docs.microsoft.com/en-us/sysinternals/downloads/procdump" title="procdump">procdump</a> to display the exception.</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">Procdump <span class="literal">-e</span> <span class="number">1</span> <span class="operator">-f</span> <span class="string">""</span> <PID of w3wp.exe></span><br></pre></td></tr></table></figure>
<p>Below is an example output.</p>
<blockquote>
<p>[00:28:10] Exception: E0434F4D.System.Configuration.ConfigurationErrorsException (“This element is not currently associated with any context”)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (“The SELECT permission was denied on the object ‘Settings’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator.Internal’.”)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (“The SELECT permission was denied on the object ‘Settings’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator.Internal’.”)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (<font color=#ff7f50><strong>“The SELECT permission was denied on the object ‘Settings’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator.Internal’.”</strong></font>)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (“The SELECT permission was denied on the object ‘Settings’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator.Internal’.”)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (“The EXECUTE permission was denied on the object ‘GetSecurityToken’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator’.”)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (“The EXECUTE permission was denied on the object ‘GetSecurityToken’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator’.”)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (<font color=#ff7f50><strong>“The EXECUTE permission was denied on the object ‘GetSecurityToken’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator’.”</strong></font>)<br>[00:28:10] Exception: E0434F4D.System.Data.SqlClient.SqlException (“The EXECUTE permission was denied on the object ‘GetSecurityToken’, database ‘Orchestrator’, schema ‘Microsoft.SystemCenter.Orchestrator’.”)<br>[00:28:10] Exception: E0434F4D.System.Data.EntityCommandExecutionException (<font color=#1F9F3E><strong>“An error occurred while executing the command definition. See the inner exception for details.”</strong></font>)<br>[00:28:10] Exception: E0434F4D.System.Data.EntityCommandExecutionException (“An error occurred while executing the command definition. See the inner exception for details.”)<br>[00:28:10] Exception: E0434F4D.System.Data.EntityCommandExecutionException (“An error occurred while executing the command definition. See the inner exception for details.”)<br>[00:28:10] Exception: E0434F4D.System.Data.EntityCommandExecutionException (“An error occurred while executing the command definition. See the inner exception for details.”)<br>[00:28:10] Exception: E0434F4D.System.Data.EntityCommandExecutionException (“An error occurred while executing the command definition. See the inner exception for details.”)</p>
</blockquote>
<p>In this example, the solution will be as shown in <strong>Troubleshooting - 2</strong>.</p>
<h3 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h3><p><a target="_blank" rel="noopener" href="https://sc-orchestrator.vasol.eu/request-error-while-opening-orchestrator-web-services/" title="Request Error – while Opening Orchestrator Web Services">Request Error – while Opening Orchestrator Web Services</a></p>
<p><a target="_blank" rel="noopener" href="https://support.microsoft.com/en-us/help/2919041/the-orchestration-console-cannot-be-opened-after-upgrading-to-system-c" title="The Orchestration Console cannot be opened after upgrading to System Center 2012 R2">The Orchestration Console cannot be opened after upgrading to System Center 2012 R2</a></p>
<p><a target="_blank" rel="noopener" href="https://blogs.technet.microsoft.com/silvana/2016/08/11/orchestrator-orchestration-console-and-web-service-are-not-working-anymore/" title="[Orchestrator] Orchestration Console and Web Service are not working anymore">[Orchestrator] Orchestration Console and Web Service are not working anymore</a></p>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2019/11/24/sco_web_fail/" data-id="cuidViVdSkG3Ez6ff6HbwSWxh" data-title="Fail to connect to Orchestrator Web Service" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Connection/" rel="tag">Connection</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCORCH/" rel="tag">SCORCH</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Web-Service/" rel="tag">Web Service</a></li></ul>
</footer>
</div>
</article>
<article id="post-scom_web_console" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2019/11/16/scom_web_console/" class="article-date">
<time class="dt-published" datetime="2019-11-15T16:00:00.000Z" itemprop="datePublished">2019-11-16</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/System-Center-Operations-Manager/">System Center Operations Manager</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2019/11/16/scom_web_console/">SCOM Web Console – "Web Console Configuration Required"</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<p>When opening SCOM Web Console, you may get the notification <font color=#ff7f50><em>“Web Console Configuration Required”</em>.</font><br><img src="/../images/scom6.png"></p>
<p>Normally you can click the <strong>Configure</strong> button (with SCOM Admin permission) and run the executable and refresh the browser window and you should see a web console login page next.</p>
<p>If you have run <strong>SilverlightClientConfiguration.exe</strong> but the problem still persists, you may want to continue with this article.</p>
<h3 id="How-to-resolve-the-notification"><a href="#How-to-resolve-the-notification" class="headerlink" title="How to resolve the notification?"></a>How to resolve the notification?</h3><p>To find the way to resolve the notification, first we need to understand what <strong>SilverlightClientConfiguration.exe</strong> acutally does.</p>
<p><strong>SilverlightClientConfiguration.exe</strong> actually does two things:</p>
<ol>
<li>Import the certificate to the store “Trusted Publishers”.</li>
<li>Add the registry.</li>
</ol>
<figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">Path: HKLM\SOFTWARE\Wow6432Node\Microsoft\Silverlight</span><br><span class="line">Value Type: Reg_DWORD</span><br><span class="line">Key: AllowElevatedTrustAppsInBrowser</span><br><span class="line">Value: 1</span><br></pre></td></tr></table></figure>
<p>Now we know the story behind, we can do the same manually if the executable doesn’t work.</p>
<p>For more details, please refer to the article <a target="_blank" rel="noopener" href="https://www.scom2k7.com/scom-2012-web-console-configuration-no-longer-required/">SCOM 2012 Web Console Configuration “NO LONGER Required!”</a>.</p>
<h3 id="It-is-a-knonw-issue-in-SCOM-2012-R2-UR12-and-UR13"><a href="#It-is-a-knonw-issue-in-SCOM-2012-R2-UR12-and-UR13" class="headerlink" title="It is a knonw issue in SCOM 2012 R2 UR12 and UR13"></a>It is a knonw issue in SCOM 2012 R2 UR12 and UR13</h3><p>We have a known issue in SCOM 2012 R2 UR13 and UR14 that the dialogue will occur again after clicking <strong>Configure</strong>. In the two versions, <strong>SilverlightClientConfiguration.exe</strong> doesn’t import the certificate successfully.</p>
<h4 id="Workaround"><a href="#Workaround" class="headerlink" title="Workaround"></a>Workaround</h4><p>There is a workaround to resolve the issue. That must be applied to every machine where you want to open web console. The steps are actually a manual implementation of <strong>SilverlightClientConfiguration.exe</strong>.</p>
<ol>
<li>Click <strong>Configure</strong> in the dialog box.</li>
<li>When you are prompted to run or save the <strong>SilverlightClientConfiguration.exe</strong> file, click <strong>Save</strong>.</li>
<li>If you never ran <strong>SilverlightClientConfiguration.exe</strong> on the machine before, run it.</li>
<li>Right-click the <strong>.exe</strong> file, click <strong>Properties</strong>, and then select the <strong>Digital Signatures</strong> tab.</li>
<li>Select the certificate that has <strong>Digest Algorithm</strong> as <strong>SHA256</strong>, and then click <strong>Details</strong>.</li>
<li>In the <strong>Digital Signature Details</strong> dialog box, click <strong>View Certificate</strong>.</li>
<li>In the dialog box that appears, click <strong>Install Certificate</strong>.</li>
<li>In the <strong>Certificate Import Wizard</strong>, change the store location to <strong>Local Machine</strong>, and then click <strong>Next</strong>.</li>
<li>Select the <strong>Place all certificates in the following store</strong> option and then select <strong>Trusted Publishers</strong>.</li>
<li>Click <strong>Next</strong> and then click <strong>Finish</strong>.</li>
<li>Refresh your browser window.</li>
</ol>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2019/11/16/scom_web_console/" data-id="cuid8O6Rt3mrQPnEjxMG-dlYj" data-title="SCOM Web Console – "Web Console Configuration Required"" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCOM/" rel="tag">SCOM</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Web-Console/" rel="tag">Web Console</a></li></ul>
</footer>
</div>
</article>
<article id="post-sco_web" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2019/11/11/sco_web/" class="article-date">
<time class="dt-published" datetime="2019-11-10T16:00:00.000Z" itemprop="datePublished">2019-11-11</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/System-Center-Orchestrator/">System Center Orchestrator</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2019/11/11/sco_web/">SCORCH Web Service API</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<h3 id="Web-Service-URLs"><a href="#Web-Service-URLs" class="headerlink" title="Web Service URLs"></a>Web Service URLs</h3><p><strong>Web Service URL</strong> (no change for 2016):</p>
<p><code>http://sco01:81/orchestrator2012/orchestrator.svc/</code> </p>
<p><strong>Web Console URL</strong>:</p>
<p><code>http://sco01:82/</code></p>
<p><strong>Get runbooks</strong>:</p>
<p><code>http://server01.contoso.com:81/Orchestrator2012/Orchestrator.svc/Runbooks</code></p>
<h3 id="Access-Orchestrator-Runbooks-via-Web-Service"><a href="#Access-Orchestrator-Runbooks-via-Web-Service" class="headerlink" title="Access Orchestrator Runbooks via Web Service"></a>Access Orchestrator Runbooks via Web Service</h3><h4 id="Get-a-Runbook"><a href="#Get-a-Runbook" class="headerlink" title="Get a Runbook"></a>Get a Runbook</h4><h5 id="PowerShell"><a href="#PowerShell" class="headerlink" title="PowerShell"></a>PowerShell</h5><figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$secpasswd</span> = <span class="built_in">ConvertTo-SecureString</span> <span class="string">"Password01!"</span> <span class="literal">-AsPlainText</span> <span class="literal">-Force</span></span><br><span class="line"><span class="variable">$mycreds</span> = <span class="built_in">New-Object</span> System.Management.Automation.PSCredential(<span class="string">"contoso\Administrator"</span>, <span class="variable">$secpasswd</span>)</span><br><span class="line"><span class="variable">$OrchURI</span> = <span class="string">"http://SCO01:81/Orchestrator2012/Orchestrator.svc/Runbooks?`$filter=Name eq 'Test'"</span></span><br><span class="line"><span class="variable">$ResponseObject</span> = <span class="built_in">invoke-webrequest</span> <span class="literal">-Uri</span> <span class="variable">$OrchURI</span> <span class="literal">-method</span> Get <span class="literal">-Credential</span> <span class="variable">$mycreds</span></span><br><span class="line"><span class="variable">$ResponseObject</span>.Content</span><br></pre></td></tr></table></figure>
<p><img src="/../images/sco2.png"></p>
<h5 id="Postman"><a href="#Postman" class="headerlink" title="Postman"></a>Postman</h5><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">http://SCO01:81/Orchestrator2012/Orchestrator.svc/Runbooks?$filter=Name eq 'Test'"</span><br></pre></td></tr></table></figure>
<p><img src="/../images/sco3.png"></p>
<h4 id="Start-a-Runbook"><a href="#Start-a-Runbook" class="headerlink" title="Start a Runbook"></a>Start a Runbook</h4><h5 id="PowerShell-1"><a href="#PowerShell-1" class="headerlink" title="PowerShell"></a>PowerShell</h5><figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$OrchURI</span> = <span class="string">"http://SCO01:81/Orchestrator2012/Orchestrator.svc/Jobs/"</span></span><br><span class="line"> </span><br><span class="line"><span class="variable">$POSTBody</span> = <span class="string">@"</span></span><br><span class="line"><span class="string"><?xml version="1.0" encoding="utf-8" standalone="yes"?></span></span><br><span class="line"><span class="string"><entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom"></span></span><br><span class="line"><span class="string"><content type="application/xml"></span></span><br><span class="line"><span class="string"><m:properties></span></span><br><span class="line"><span class="string"><d:RunbookId type="Edm.Guid">87be7221-2a15-4e41-91f6-35f4d3006d53</d:RunbookId></span></span><br><span class="line"><span class="string"><d:Parameters></d:Parameters></span></span><br><span class="line"><span class="string"></m:properties></span></span><br><span class="line"><span class="string"></content></span></span><br><span class="line"><span class="string"></entry></span></span><br><span class="line"><span class="string">"@</span></span><br><span class="line"> </span><br><span class="line"><span class="variable">$ResponseObject</span> = <span class="built_in">invoke-webrequest</span> <span class="literal">-Uri</span> <span class="variable">$OrchURI</span> <span class="literal">-method</span> POST <span class="literal">-Credential</span> <span class="variable">$mycreds</span> <span class="literal">-Body</span> <span class="variable">$POSTBody</span> <span class="literal">-ContentType</span> <span class="string">"application/atom+xml"</span></span><br><span class="line"><span class="variable">$ResponseObject</span>.Content</span><br></pre></td></tr></table></figure>
<p><img src="/../images/sco4.png"></p>
<h5 id="Postman-1"><a href="#Postman-1" class="headerlink" title="Postman"></a>Postman</h5><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">http://SCO01:81/Orchestrator2012/Orchestrator.svc/Jobs</span><br></pre></td></tr></table></figure>
<p><img src="/../images/sco5.png"></p>
<p><img src="/../images/sco6.png"></p>
<p><img src="/../images/sco7.png"></p>
<p>Confirm the runbook enters running status:</p>
<p><img src="/../images/sco8.png"></p>
<h3 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h3><p><a target="_blank" rel="noopener" href="https://techcommunity.microsoft.com/t5/System-Center-Blog/Accessing-System-Center-2012-Orchestrator-Using-the-Web-Service/ba-p/345014" title="Accessing System Center 2012 Orchestrator Using the Web Service">Accessing System Center 2012 Orchestrator Using the Web Service</a></p>
<p><a target="_blank" rel="noopener" href="https://techcommunity.microsoft.com/t5/System-Center-Blog/Starting-Runbooks-and-Stopping-Jobs-Using-the-System-Center-2012/ba-p/345024" title="Starting Runbooks and Stopping Jobs Using the System Center 2012 Orchestrator Web Service">Starting Runbooks and Stopping Jobs Using the System Center 2012 Orchestrator Web Service</a></p>
<p><a target="_blank" rel="noopener" href="http://www.laurierhodes.info/?q=node/101" title="Calling Orchestrator Runbooks (& retrieving output) via REST">Calling Orchestrator Runbooks (& retrieving output) via REST</a></p>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2019/11/11/sco_web/" data-id="cuid1nibJxwog2fzUgiO1FyYF" data-title="SCORCH Web Service API" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Postman/" rel="tag">Postman</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/PowerShell/" rel="tag">PowerShell</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Rest-API/" rel="tag">Rest API</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCORCH/" rel="tag">SCORCH</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Web-Service/" rel="tag">Web Service</a></li></ul>
</footer>
</div>
</article>
<article id="post-scom_gw_dmz" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2019/11/07/scom_gw_dmz/" class="article-date">
<time class="dt-published" datetime="2019-11-06T16:00:00.000Z" itemprop="datePublished">2019-11-07</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/System-Center-Operations-Manager/">System Center Operations Manager</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2019/11/07/scom_gw_dmz/">Troubleshooting Gateway or Workgroup Communication Issue in SCOM</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<p>If you have deployed SCOM Gateway in your environment, or SCOM agent on a workgroup machine, but they can’t communicate with management server, you can refer to the following steps for troubleshooting.</p>
<p>Based on my experience, these troubleshooting steps can help resolve >50% of Gateway/Workgroup communication issues.</p>
<h3 id="Troubleshooting-Steps"><a href="#Troubleshooting-Steps" class="headerlink" title="Troubleshooting Steps"></a>Troubleshooting Steps</h3><ul>
<li><p>Make sure <strong>Microsoft Monitoring Agent</strong> service is running, and Startup type is set to <strong>Automatic</strong> on management server and gateway/workgroup.</p>
</li>
<li><p>Check the gateway/workgroup status in SCOM console.</p>
</li>
<li><p><strong>Check Operations Manager event log</strong></p>
<p> If the certificate is recreated, we should find Event ID 20053 in Operations Manager event log on both the management server and gateway server/workgroup, indicating the certificate was loaded successfully.</p>
</li>
</ul>
<p><img src="/../images/scom1.png"></p>
<pre><code>If you find a related warning or error in event log, refer to this blog for explanation: [Troubleshoot workgroup/gateway issue with event log](https://techcommunity.microsoft.com/t5/System-Center-Blog/Monitoring-OpsMgr-workgroup-clients-Part-2-Installing/ba-p/350806)
</code></pre>
<ul>
<li><p><strong>Ensure port 5723 is open</strong></p>
<p> On gateway/workgroup:</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">telnet <Management Server FQDN> <span class="number">5723</span></span><br></pre></td></tr></table></figure>
</li>
<li><p><strong>DNS</strong></p>
<p> Make sure that both sides are reachable via hostname (just ping). If it’s not working add the computers in DNS or in the <strong>Host</strong> file (<strong>C:\Windows\System32\Drivers\ETC\Host</strong>).</p>
</li>
<li><p><strong>Check certificate</strong></p>
<p> Computer Certificate:</p>
<ol>
<li>Go to both management server and gateway server/workgroup, navigate to <strong>HKLM\Software\Microsoft\Microsoft Operations Manager\3.0\Machine Settings</strong> and check the value of <strong>ChannelCertificateSerialNumber</strong>.</li>
</ol>
<p> <img src="/../images/scom2.png"></p>
<p> Then, open <strong>mmc</strong> -> <strong>File</strong> -> <strong>Add or Remove Snap-ins</strong> -> <strong>Certificates</strong> -> <strong>Add</strong>, select <strong>Computer account</strong>, then <strong>Next</strong> and <strong>Finish</strong>. In <strong>Personal</strong> -> <strong>Certificates</strong>, open the certificate that you install on management server/gateway server/workgroup agent, click on <strong>Details</strong> tab and check the <strong>Serial Number</strong>. The <strong>Serial number</strong> and the registry value <strong>ChannelCertificateSerialNumber</strong> should match each other.</p>
<p> <img src="/../images/scom3.png"></p>
<ol start="2">
<li><p>Confirm computer certificates on both sides are issued by the same CA.</p>
</li>
<li><p>The certificate must includes the private key.</p>
</li>
</ol>
<p> <img src="/../images/scom4.png"></p>
<ol start="4">
<li>The certificate must be trusted all the way to the root (Chain)</li>
</ol>
<p> <img src="/../images/scom5.png"></p>
<ol start="5">
<li><p>The <strong>Common Name (CN)</strong> value in the certificate’s <strong>Subject</strong> field must match the FQDN of the computer where you imported the certificate.</p>
</li>
<li><p>Ensure the certificate includes both Server Authentication OID and Client Authentication OID in the Enhanced Key Usage property.</p>
</li>
<li><p>Check expiration date.</p>
</li>
<li><p>Other comments:<br> Hash Algorithm does not need to be same between source and target. (Ex. SHA1 on MS and SHA256 on GTW works)<br> Key size can be 2048 and 4096.</p>
</li>
</ol>
<p> Trusted Root Certificate:</p>
<p> Confirm trusted Root Certificate is well imported and can be found under <strong>Trusted Root Certification Authorities</strong> – <strong>Certificates</strong>. Check expiration date.</p>
</li>
<li><p>On the gateway server, go to <strong>HKLM\Software\ Microsoft\Microsoft Operation Manager\3.0\Server Management Group<MG Name>\Parent Health Services\0</strong>. Ensure the <strong>AuthenticationName</strong> and the <strong>NetworkName</strong> match and is FQDN of management server.</p>
<p> In workgroup scenario, the path is <strong>HKLM\Software\ Microsoft\Microsoft Operation Manager\3.0\Agent Management Group<MG Name>\Parent Health Services\0</strong></p>
</li>
<li><p>Ensure the assigned management server in Control Panel is correct.</p>
</li>
<li><p><strong>Duplicate SPN</strong></p>
<p> On management server, identify the duplicate SPN. Any duplicate SPN’s will be listed.</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">setspn <span class="literal">-x</span></span><br></pre></td></tr></table></figure>
<p> Delete the duplicate SPN</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">setspn –d <SPN> <object></span><br></pre></td></tr></table></figure>
<p> Example:</p>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">setspn <span class="literal">-D</span> http/daserver daserver1</span><br><span class="line">setspn –d host/fscluster member</span><br></pre></td></tr></table></figure></li>
</ul>
<h3 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h3><p><a target="_blank" rel="noopener" href="http://blog.ctglobalservices.com/operations-manager-scom/msk/common-issues-when-working-with-certificates-in-opsmgr/">Common issues when working with certificates in OpsMgr</a></p>
<p><a target="_blank" rel="noopener" href="https://support.microsoft.com/en-us/help/947691/when-you-try-to-install-a-system-center-operations-manager-agent-on-a">When you try to install a System Center Operations Manager agent on a workgroup computer without using a gateway server, Operations Manager cannot see the workgroup computer</a></p>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2019/11/07/scom_gw_dmz/" data-id="cuidLYpyJ2mcmXaZKoxGOM2wk" data-title="Troubleshooting Gateway or Workgroup Communication Issue in SCOM" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Certificate/" rel="tag">Certificate</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Communication/" rel="tag">Communication</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Gateway/" rel="tag">Gateway</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Operations-Manager/" rel="tag">Operations Manager</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCOM/" rel="tag">SCOM</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Workgroup/" rel="tag">Workgroup</a></li></ul>
</footer>
</div>
</article>
<article id="post-scsm_ad_issue" class="h-entry article article-type-post" itemprop="blogPost" itemscope itemtype="https://schema.org/BlogPosting">
<div class="article-meta">
<a href="/2019/09/30/scsm_ad_issue/" class="article-date">
<time class="dt-published" datetime="2019-09-29T16:00:00.000Z" itemprop="datePublished">2019-09-30</time>
</a>
<div class="article-category">
<a class="article-category-link" href="/categories/System-Center-Service-Manager/">System Center Service Manager</a>
</div>
</div>
<div class="article-inner">
<header class="article-header">
<h1 itemprop="name">
<a class="p-name article-title" href="/2019/09/30/scsm_ad_issue/">Changes of user relationship in AD can’t be correctly synchronized to SCSM</a>
</h1>
</header>
<div class="e-content article-entry" itemprop="articleBody">
<h3 id="Issue-Description"><a href="#Issue-Description" class="headerlink" title="Issue Description"></a>Issue Description</h3><p>The relationship of users might be not correctly updated if related users were once re-named in AD or have duplicate records in DB.</p>
<h3 id="Cause-1-–-Known-Issue-a-renamed-user-in-AD-will-be-treated-as-a-new-object-in-SCSM"><a href="#Cause-1-–-Known-Issue-a-renamed-user-in-AD-will-be-treated-as-a-new-object-in-SCSM" class="headerlink" title="Cause 1 – Known Issue: a renamed user in AD will be treated as a new object in SCSM"></a>Cause 1 – Known Issue: a renamed user in AD will be treated as a new object in SCSM</h3><h4 id="Symptom"><a href="#Symptom" class="headerlink" title="Symptom"></a>Symptom</h4><p>There are multiple records for the same user in BaseManagedEntity.</p>
<h4 id="Analysis"><a href="#Analysis" class="headerlink" title="Analysis"></a>Analysis</h4><p>There is a <a target="_blank" rel="noopener" href="https://blogs.technet.microsoft.com/manageabilityguys/2013/06/17/managing-username-changes-in-service-manager/" title="blog">blog</a> talking about this kind of behavior.</p>
<p>I did a test in my lab environment (SCSM 2012 R2). After I rename a user in AD and run AD connector, there will be two objects co-existing in the table <strong>[dbo].[ManagedEntity]</strong> for the user, and two relationships in the table <strong>[dbo].[Relationship]</strong>.</p>
<p>For example, after I change my user name from “wendi” to “wendii”, then to “wendie”, there are 3 user objects and 3 relationships in the DB.</p>
<p>Relationship:</p>
<p><img src="/../images/scsm1.png"></p>
<p>BaseManagedEntity:</p>
<p><img src="/../images/scsm2.png"></p>
<p>Then I manually deleted “wendi” from DB, and changed the manager from “weiwen” to “Administrator” in AD. The relationship got updated for the newest object “wendie”, but not for the old object “wendii”. That resulted in two managers for the user “wendii”/”wendie”, which is actually the same user in AD.</p>
<p><img src="/../images/scsm3.png"></p>
<h4 id="Resolution"><a href="#Resolution" class="headerlink" title="Resolution"></a>Resolution</h4><p>Firstly remove the duplicated users from <strong>[dbo].[ManagedEntity]</strong>.</p>
<ol>
<li>If you don’t want to lose the relationships associated with the old object, please use the script in the blog to move all relationships from old object to new object.</li>
</ol>
<p><strong>Note</strong>: In the script there is a path pointing to SCSM PowerShell Module. You may need to alter it manually based on the real location.</p>
<ol start="2">
<li>Remove the duplicate user object:</li>
</ol>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$oldADUser</span> = <span class="string">"wendi"</span></span><br><span class="line"><span class="built_in">Get-SCClassInstance</span> <span class="literal">-Class</span> (<span class="built_in">Get-SCClass</span> <span class="literal">-Name</span> <span class="string">"System.Domain.User"</span>) <span class="literal">-Filter</span> <span class="string">"UserName -eq <span class="variable">$oldADUser</span> | Remove-SCClassInstance</span></span><br></pre></td></tr></table></figure>
<p>After confirming there is no duplicate user objects, please change the manager relationship in AD, then check if the relationship can be updated correctly by AD connector.</p>
<h3 id="Cause-2-–-Duplicate-AD-connector-introduces-duplicate-user-records"><a href="#Cause-2-–-Duplicate-AD-connector-introduces-duplicate-user-records" class="headerlink" title="Cause 2 – Duplicate AD connector introduces duplicate user records"></a>Cause 2 – Duplicate AD connector introduces duplicate user records</h3><p>If there is only 1 record in BaseManagedEntity, but multiple records in <strong>[LFXSTG].[AD_User]</strong>, it is probably from duplicate AD connectors.</p>
<p>We can check the data sources of the records in <strong>[LFXSTG].[AD_User]</strong>:</p>
<p><img src="/../images/scsm4.png"></p>
<p>This query can give you information about all data sources (connectors):</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">Select</span> <span class="operator">*</span> <span class="keyword">from</span> LFX.Datasource</span><br></pre></td></tr></table></figure>
<p>A sample output:</p>
<p><img src="/../images/scsm5.png"></p>
<h4 id="Resolution-1"><a href="#Resolution-1" class="headerlink" title="Resolution"></a>Resolution</h4><p>Disable all duplicate connectors. After that, all users modified in the future could be correctly updated.</p>
<p>For those users which are already affected by the issue, we can follow below steps to process them.</p>
<ol>
<li>Run this query in ServiceManager to get the users who have duplicate relationships with relationship isDeleted = 0, as well as the connectors that brought the relationships.</li>
</ol>
<p>(This query focuses on “manager” relationship)</p>
<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">Select</span> <span class="keyword">distinct</span></span><br><span class="line">u.DisplayName <span class="string">'User Display Name'</span>,</span><br><span class="line">u.UserName_6AF77E23_669B_123F_B392_323C17097BBD <span class="string">'User'</span>, </span><br><span class="line">Manager.UserName_6AF77E23_669B_123F_B392_323C17097BBD <span class="string">'Manager'</span>, </span><br><span class="line">r.RelationshipId, </span><br><span class="line">r.IsDeleted <span class="string">'Is Relationship Deleted'</span>,</span><br><span class="line">C.DisplayName <span class="string">'Connector'</span>,</span><br><span class="line">BME.IsDeleted <span class="string">'Is Connector Deleted'</span>,</span><br><span class="line">R.LastModified</span><br><span class="line"><span class="keyword">from</span> Relationship R</span><br><span class="line"><span class="keyword">left</span> <span class="keyword">join</span> RelationshipType RT <span class="keyword">on</span> R.RelationshipTypeId <span class="operator">=</span> RT.RelationshipTypeId</span><br><span class="line"><span class="keyword">left</span> <span class="keyword">join</span> MT_System$Domain$<span class="keyword">User</span> Manager <span class="keyword">on</span> manager.BaseManagedEntityId <span class="operator">=</span> R.SourceEntityId</span><br><span class="line"><span class="keyword">left</span> <span class="keyword">join</span> MT_System$Domain$<span class="keyword">User</span> U <span class="keyword">on</span> u.BaseManagedEntityId <span class="operator">=</span> r.TargetEntityId</span><br><span class="line"><span class="keyword">inner</span> <span class="keyword">join</span> DiscoverySourceToRelationship DSTR <span class="keyword">on</span> R.RelationshipId <span class="operator">=</span> DSTR.RelationshipId</span><br><span class="line"><span class="keyword">Left</span> <span class="keyword">join</span> DiscoverySource DS <span class="keyword">on</span> DS.DiscoverySourceId <span class="operator">=</span> DSTR.DiscoverySourceId</span><br><span class="line"><span class="keyword">left</span> <span class="keyword">join</span> MT_Connector C <span class="keyword">on</span> <span class="keyword">convert</span>(nvarchar(<span class="number">256</span>),DS.ConnectorId) <span class="operator">=</span> C.Id</span><br><span class="line"><span class="keyword">Left</span> <span class="keyword">join</span> BaseManagedEntity BME <span class="keyword">on</span> C.BaseManagedEntityId <span class="operator">=</span> BME.BaseManagedEntityId</span><br><span class="line"><span class="keyword">where</span> RelationshipTypeName <span class="keyword">like</span> <span class="string">'%System.UserManagesUser%'</span> <span class="keyword">and</span> u.BaseManagedEntityId <span class="keyword">in</span></span><br><span class="line">(</span><br><span class="line"><span class="keyword">Select</span> </span><br><span class="line">u.BaseManagedEntityId</span><br><span class="line"><span class="keyword">from</span> Relationship R</span><br><span class="line"><span class="keyword">left</span> <span class="keyword">join</span> RelationshipType RT <span class="keyword">on</span> R.RelationshipTypeId <span class="operator">=</span> RT.RelationshipTypeId</span><br><span class="line"><span class="keyword">left</span> <span class="keyword">join</span> MT_System$Domain$<span class="keyword">User</span> Manager <span class="keyword">on</span> manager.BaseManagedEntityId <span class="operator">=</span> R.SourceEntityId</span><br><span class="line"><span class="keyword">left</span> <span class="keyword">join</span> MT_System$Domain$<span class="keyword">User</span> U <span class="keyword">on</span> u.BaseManagedEntityId <span class="operator">=</span> r.TargetEntityId</span><br><span class="line"><span class="keyword">where</span> RelationshipTypeName <span class="keyword">like</span> <span class="string">'%System.UserManagesUser%'</span> <span class="keyword">and</span> r.IsDeleted <span class="operator">=</span> <span class="number">0</span></span><br><span class="line"><span class="keyword">group</span> <span class="keyword">by</span> U.UserName_6AF77E23_669B_123F_B392_323C17097BBD, u.BaseManagedEntityId</span><br><span class="line"><span class="keyword">Having</span> <span class="built_in">count</span>(u.UserName_6AF77E23_669B_123F_B392_323C17097BBD) <span class="operator">></span> <span class="number">1</span>)</span><br><span class="line"><span class="keyword">order</span> <span class="keyword">by</span> U.UserName_6AF77E23_669B_123F_B392_323C17097BBD</span><br></pre></td></tr></table></figure>
<ol start="2">
<li>Use below steps to automate the removal of un-needed relationships whilst keeping the most current one.</li>
</ol>
<ul>
<li>Copy the query result with headers and save as a .csv file.</li>
<li>Remove the needed relationships from the .csv file.</li>
<li>Use below PowerShell commands to remove the un-needed relationships.</li>
</ul>
<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># You may change the file path.</span></span><br><span class="line"><span class="variable">$listcsv</span> = <span class="built_in">Import-Csv</span> C:\Files\UnneededRelationships.csv</span><br><span class="line"><span class="keyword">foreach</span>(<span class="variable">$list</span> <span class="keyword">in</span> <span class="variable">$listcsv</span>)</span><br><span class="line">{</span><br><span class="line"> <span class="built_in">Get-screlationshipinstance</span> <span class="literal">-id</span> <span class="variable">$listcsv</span>.RelationshipId | <span class="built_in">remove-screlationshipinstance</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure>
</div>
<footer class="article-footer">
<a data-url="http://windyztree.github.io/2019/09/30/scsm_ad_issue/" data-id="cuidBiJra9_zAcR1pis5LNoY3" data-title="Changes of user relationship in AD can’t be correctly synchronized to SCSM" class="article-share-link"><span class="fa fa-share">Share</span></a>
<ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/AD-Connector/" rel="tag">AD Connector</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Connector/" rel="tag">Connector</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/Relationship/" rel="tag">Relationship</a></li><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/SCSM/" rel="tag">SCSM</a></li></ul>
</footer>
</div>
</article>
<nav id="page-nav">
<span class="page-number current">1</span><a class="page-number" href="/page/2/">2</a><a class="extend next" rel="next" href="/page/2/">Next »</a>
</nav>
</section>
<aside id="sidebar">
<div class="widget-wrap">
<h3 class="widget-title">Categories</h3>
<div class="widget">
<ul class="category-list"><li class="category-list-item"><a class="category-list-link" href="/categories/AI/">AI</a></li><li class="category-list-item"><a class="category-list-link" href="/categories/Kubernetes/">Kubernetes</a></li><li class="category-list-item"><a class="category-list-link" href="/categories/System-Center-Operations-Manager/">System Center Operations Manager</a></li><li class="category-list-item"><a class="category-list-link" href="/categories/System-Center-Orchestrator/">System Center Orchestrator</a></li><li class="category-list-item"><a class="category-list-link" href="/categories/System-Center-Service-Manager/">System Center Service Manager</a></li><li class="category-list-item"><a class="category-list-link" href="/categories/%E7%BB%BC%E5%90%88/">综合</a></li></ul>
</div>
</div>
<div class="widget-wrap">
<h3 class="widget-title">Tags</h3>
<div class="widget">
<ul class="tag-list" itemprop="keywords"><li class="tag-list-item"><a class="tag-list-link" href="/tags/AD-Connector/" rel="tag">AD Connector</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/AI-Agent/" rel="tag">AI Agent</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/CRI-Dockerd/" rel="tag">CRI-Dockerd</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Certificate/" rel="tag">Certificate</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Cluster/" rel="tag">Cluster</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Communication/" rel="tag">Communication</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Connection/" rel="tag">Connection</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Connector/" rel="tag">Connector</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Deployment/" rel="tag">Deployment</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Gateway/" rel="tag">Gateway</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Heartbeat/" rel="tag">Heartbeat</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Integration-Pack/" rel="tag">Integration Pack</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Kubernetes/" rel="tag">Kubernetes</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Memory/" rel="tag">Memory</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/MySQL/" rel="tag">MySQL</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Network/" rel="tag">Network</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Nginx/" rel="tag">Nginx</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Openclaw/" rel="tag">Openclaw</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Operations-Manager/" rel="tag">Operations Manager</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/PHP/" rel="tag">PHP</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Postman/" rel="tag">Postman</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/PowerShell/" rel="tag">PowerShell</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Relationship/" rel="tag">Relationship</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Rest-API/" rel="tag">Rest API</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/SCOM/" rel="tag">SCOM</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/SCORCH/" rel="tag">SCORCH</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/SCSM/" rel="tag">SCSM</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/SPN/" rel="tag">SPN</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/SQL-Discovery/" rel="tag">SQL Discovery</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Skill/" rel="tag">Skill</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Ubuntu/" rel="tag">Ubuntu</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Upgrade/" rel="tag">Upgrade</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Web-Console/" rel="tag">Web Console</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Web-Service/" rel="tag">Web Service</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/WordPress/" rel="tag">WordPress</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/Workgroup/" rel="tag">Workgroup</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8/" rel="tag">云服务器</a></li><li class="tag-list-item"><a class="tag-list-link" href="/tags/%E5%8D%9A%E5%AE%A2/" rel="tag">博客</a></li></ul>
</div>
</div>
<div class="widget-wrap">
<h3 class="widget-title">Tag Cloud</h3>
<div class="widget tagcloud">
<a href="/tags/AD-Connector/" style="font-size: 10px;">AD Connector</a> <a href="/tags/AI-Agent/" style="font-size: 10px;">AI Agent</a> <a href="/tags/CRI-Dockerd/" style="font-size: 10px;">CRI-Dockerd</a> <a href="/tags/Certificate/" style="font-size: 10px;">Certificate</a> <a href="/tags/Cluster/" style="font-size: 10px;">Cluster</a> <a href="/tags/Communication/" style="font-size: 10px;">Communication</a> <a href="/tags/Connection/" style="font-size: 15px;">Connection</a> <a href="/tags/Connector/" style="font-size: 10px;">Connector</a> <a href="/tags/Deployment/" style="font-size: 10px;">Deployment</a> <a href="/tags/Gateway/" style="font-size: 10px;">Gateway</a> <a href="/tags/Heartbeat/" style="font-size: 10px;">Heartbeat</a> <a href="/tags/Integration-Pack/" style="font-size: 10px;">Integration Pack</a> <a href="/tags/Kubernetes/" style="font-size: 15px;">Kubernetes</a> <a href="/tags/Memory/" style="font-size: 10px;">Memory</a> <a href="/tags/MySQL/" style="font-size: 10px;">MySQL</a> <a href="/tags/Network/" style="font-size: 10px;">Network</a> <a href="/tags/Nginx/" style="font-size: 10px;">Nginx</a> <a href="/tags/Openclaw/" style="font-size: 10px;">Openclaw</a> <a href="/tags/Operations-Manager/" style="font-size: 10px;">Operations Manager</a> <a href="/tags/PHP/" style="font-size: 10px;">PHP</a> <a href="/tags/Postman/" style="font-size: 10px;">Postman</a> <a href="/tags/PowerShell/" style="font-size: 10px;">PowerShell</a> <a href="/tags/Relationship/" style="font-size: 10px;">Relationship</a> <a href="/tags/Rest-API/" style="font-size: 10px;">Rest API</a> <a href="/tags/SCOM/" style="font-size: 20px;">SCOM</a> <a href="/tags/SCORCH/" style="font-size: 20px;">SCORCH</a> <a href="/tags/SCSM/" style="font-size: 10px;">SCSM</a> <a href="/tags/SPN/" style="font-size: 10px;">SPN</a> <a href="/tags/SQL-Discovery/" style="font-size: 10px;">SQL Discovery</a> <a href="/tags/Skill/" style="font-size: 10px;">Skill</a> <a href="/tags/Ubuntu/" style="font-size: 10px;">Ubuntu</a> <a href="/tags/Upgrade/" style="font-size: 10px;">Upgrade</a> <a href="/tags/Web-Console/" style="font-size: 10px;">Web Console</a> <a href="/tags/Web-Service/" style="font-size: 15px;">Web Service</a> <a href="/tags/WordPress/" style="font-size: 10px;">WordPress</a> <a href="/tags/Workgroup/" style="font-size: 10px;">Workgroup</a> <a href="/tags/%E4%BA%91%E6%9C%8D%E5%8A%A1%E5%99%A8/" style="font-size: 10px;">云服务器</a> <a href="/tags/%E5%8D%9A%E5%AE%A2/" style="font-size: 10px;">博客</a>
</div>
</div>
<div class="widget-wrap">
<h3 class="widget-title">Archives</h3>