운영 대시보드 낡은 메모 정정 + 트윈 발송 차단 사유별 분해 추가

roadmap.md §2.6의 "모니터링 대시보드 UI·지연시간·오류율 아직 없음" 메모가
낡은 기록이었음을 재확인 -- GET /admin/dashboard(HTML)와 /admin/metrics의
생성 지연시간·오류율 계측은 이미 이전 Phase 1 B 커밋에서 구현돼 있었고,
같은 문서 §B 항목과도 모순되고 있었음. [~] -> [x]로 정정하고 실제 배경을
남김.

실제로 오늘 추가한 것: RuntimeMetrics.TwinSendsBlocked를 사유별
(peer_veto/group_conversation/flood_blocked/flood_detected/
escalate_check_error/escalated/autonomy_l0/autonomy_l1_unapproved/
autonomy_l2_no_whitelist_match/autonomy_unknown_level)로 분해하는
TwinSendsBlockedByReason을 추가하고, /admin/metrics에
twin_sends_blocked_by_reason으로 노출(기존 twin_sends_blocked 총합은
유지). 이미 JSON에는 있었지만 화면엔 안 보이던 escalations_by_reason과
새 필드를 /admin/dashboard에 사유별 표로 렌더링.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014YSB5PqF38raTxP5ABgr9m
This commit is contained in:
Claude 2026-08-03 07:12:34 +00:00
parent 97edabc47d
commit a1415f5ce0
No known key found for this signature in database
6 changed files with 228 additions and 64 deletions

View File

@ -201,6 +201,12 @@ const adminDashboardHTML = `<!doctype html>
.v { font-size: 1.4rem; font-weight: 700; margin-top: 4px; }
pre { background: #fff; border: 1px solid #d5e2db; border-radius: 10px; padding: 14px; overflow: auto; }
button { margin: 12px 0; padding: 8px 14px; border-radius: 8px; border: 0; background: #1f6f5b; color: #fff; cursor: pointer; }
.breakdowns { display: grid; gap: 16px; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); margin-top: 20px; }
.breakdown table { width: 100%; border-collapse: collapse; background: #fff; border: 1px solid #d5e2db; border-radius: 10px; overflow: hidden; }
.breakdown th, .breakdown td { text-align: left; padding: 8px 12px; font-size: .85rem; }
.breakdown th { background: #eaf1ed; color: #40554c; }
.breakdown tr + tr td { border-top: 1px solid #e4ede8; }
.breakdown td.n { text-align: right; font-weight: 600; }
</style>
</head>
<body>
@ -208,9 +214,25 @@ const adminDashboardHTML = `<!doctype html>
<p>Bearer ADMIN_API_TOKEN 으로 /admin/metrics 불러옵니다. 생성 지연·오류율은 프로세스 메모리 샘플입니다.</p>
<button onclick="load()">새로고침</button>
<div class="grid" id="cards"></div>
<div class="breakdowns">
<div class="breakdown">
<h2 style="margin:0 0 8px;font-size:1rem">에스컬레이션 사유별</h2>
<table><thead><tr><th>reason</th><th style="text-align:right">count</th></tr></thead><tbody id="escByReason"></tbody></table>
</div>
<div class="breakdown">
<h2 style="margin:0 0 8px;font-size:1rem">와카뷰 발송 차단 사유별</h2>
<table><thead><tr><th>reason</th><th style="text-align:right">count</th></tr></thead><tbody id="blockedByReason"></tbody></table>
</div>
</div>
<h2 style="margin-top:28px;font-size:1rem">raw JSON</h2>
<pre id="raw">loading</pre>
<script>
function renderBreakdown(elId, obj) {
const rows = Object.entries(obj || {}).sort((a, b) => b[1] - a[1]);
document.getElementById(elId).innerHTML = rows.length
? rows.map(([k, v]) => '<tr><td>'+k+'</td><td class="n">'+v+'</td></tr>').join('')
: '<tr><td colspan="2">없음</td></tr>';
}
async function load() {
const params = new URLSearchParams(location.search);
let token = params.get('token') || localStorage.ADMIN_API_TOKEN || '';
@ -237,6 +259,8 @@ const adminDashboardHTML = `<!doctype html>
];
document.getElementById('cards').innerHTML = cards.map(([k,v]) =>
'<div class="card"><div class="k">'+k+'</div><div class="v">'+v+'</div></div>').join('');
renderBreakdown('escByReason', data.escalations_by_reason);
renderBreakdown('blockedByReason', data.twin_sends_blocked_by_reason);
}
load();
</script>

View File

@ -93,10 +93,11 @@ func setupRouter(db *gorm.DB, relay *ConnectionManager, ai *AIServiceClient) *gi
if !requireAdmin(c) {
return
}
// v1-minimal (roadmap.md §2.6): only counts honestly derivable from
// the current schema. Draft-generation latency and AI-service error
// rate need a request-timing/logging layer that doesn't exist yet --
// not fabricated here, left for that future work.
// v1-minimal (roadmap.md §2.6): counts honestly derivable from the
// current schema, plus process-local draft/escalation timings kept
// in runtimeMetrics (see metrics.go) -- the latter reset on restart
// and are not a substitute for a real time-series DB, but are real
// numbers, not placeholders.
var usersTotal, humanMessages, twinMessages, escalationsTotal int64
var conversationsTotal, conversationsVetoed, invitesMinted, invitesUsed int64
db.Model(&User{}).Count(&usersTotal)
@ -149,6 +150,7 @@ func setupRouter(db *gorm.DB, relay *ConnectionManager, ai *AIServiceClient) *gi
"escalate_errors": rt.EscalateErrors,
"escalate_error_rate": rt.EscalateErrorRate,
"twin_sends_blocked": rt.TwinSendsBlocked,
"twin_sends_blocked_by_reason": rt.TwinSendsBlockedByReason,
"push_attempts": rt.PushAttempts,
"push_skipped": rt.PushSkipped,
"push_delivered": rt.PushDelivered,
@ -262,7 +264,7 @@ func setupRouter(db *gorm.DB, relay *ConnectionManager, ai *AIServiceClient) *gi
// later checks can override an earlier block.
if req.SenderMode == SenderTwin {
if conversation.TwinDisabledByPeer {
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("peer_veto")
c.JSON(http.StatusForbidden, gin.H{"detail": "상대방이 와카뷰를 거부해서 이 대화방에서는 자동 발송이 꺼져 있습니다"})
return
}
@ -272,7 +274,7 @@ func setupRouter(db *gorm.DB, relay *ConnectionManager, ai *AIServiceClient) *gi
// 무관하게 그룹 대화에서는 와카뷰 발송 자체를 막는다. 초안은
// 항상 사람이 검토해서 직접 보낸다.
if conversation.IsGroup {
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("group_conversation")
c.JSON(http.StatusForbidden, gin.H{"detail": "그룹 대화에서는 와카뷰 자동 발송이 허용되지 않습니다 -- 초안만 생성하고 사람이 직접 보내세요"})
return
}
@ -285,14 +287,14 @@ func setupRouter(db *gorm.DB, relay *ConnectionManager, ai *AIServiceClient) *gi
// 불가 지점에 있어, 어떤 자율성 레벨/화이트리스트로도 건너뛸 수
// 없다.
if conversation.TwinDisabledByFlood {
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("flood_blocked")
c.JSON(http.StatusForbidden, gin.H{"detail": "도배 감지로 이 대화방의 와카뷰 자동 발송이 일시중단되어 있습니다 -- 사후 알림에서 확인 후 재개할 수 있습니다"})
return
}
if exceeded, count := floodDetected(db, convID, req.SenderID); exceeded {
conversation.TwinDisabledByFlood = true
db.Save(&conversation)
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("flood_detected")
reason := floodReason(count)
db.Create(&EscalationLog{
UserID: req.SenderID,
@ -315,12 +317,12 @@ func setupRouter(db *gorm.DB, relay *ConnectionManager, ai *AIServiceClient) *gi
result, err := ai.checkEscalation(req.Text)
runtimeMetrics.recordEscalate(err)
if err != nil {
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("escalate_check_error")
c.JSON(http.StatusBadGateway, gin.H{"detail": "escalation gate unavailable, twin send blocked: " + err.Error()})
return
}
if result.Escalate {
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("escalated")
db.Create(&EscalationLog{
UserID: req.SenderID,
ConversationID: convID,
@ -347,23 +349,23 @@ func setupRouter(db *gorm.DB, relay *ConnectionManager, ai *AIServiceClient) *gi
switch level {
case AutonomyL0:
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("autonomy_l0")
c.JSON(http.StatusForbidden, gin.H{"detail": "L0(비서 모드)에서는 와카뷰 자동 발송이 허용되지 않습니다 -- 초안만 생성하고 사람이 직접 보내세요"})
return
case AutonomyL1:
if !req.Approved {
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("autonomy_l1_unapproved")
c.JSON(http.StatusForbidden, gin.H{"detail": "L1은 발송 전 사용자 승인이 필요합니다"})
return
}
case AutonomyL2:
if !req.Approved && !whitelistMatches(db, req.SenderID, convID, req.Text) {
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("autonomy_l2_no_whitelist_match")
c.JSON(http.StatusForbidden, gin.H{"detail": "화이트리스트에 없는 주제는 L1과 동일하게 사용자 승인이 필요합니다"})
return
}
default:
runtimeMetrics.recordTwinBlocked()
runtimeMetrics.recordTwinBlocked("autonomy_unknown_level")
c.JSON(http.StatusForbidden, gin.H{"detail": "알 수 없는 자율성 레벨이라 발송을 차단합니다"})
return
}

View File

@ -658,6 +658,11 @@ func TestVetoMissingConversation(t *testing.T) {
}
func TestAdminMetricsCountsMessagesEscalationsAndVeto(t *testing.T) {
// runtimeMetrics is a package-level in-process counter shared across the
// whole test binary (not reset per-test like the per-test sqlite db), so
// tests that assert on its exact totals must reset it first to avoid
// picking up counts left behind by earlier tests in this file.
runtimeMetrics = &RuntimeMetrics{}
server, db := setupTestServer(t)
senderID, token := mustSignup(t, server.URL, "메트릭")
@ -711,6 +716,67 @@ func TestAdminMetricsCountsMessagesEscalationsAndVeto(t *testing.T) {
if out["invites_minted"].(float64) < 1 || out["invites_used"].(float64) < 1 {
t.Fatalf("expected at least 1 minted/used invite, got %v", out)
}
// The "계좌번호 알려줄게" twin send above was blocked by the escalation
// gate, not by veto/flood/autonomy -- confirm it landed under the right
// by-reason key (roadmap.md monitoring gap: distinguish *why* a
// twin-authored send was blocked, not just a raw total).
blockedByReason := out["twin_sends_blocked_by_reason"].(map[string]interface{})
if blockedByReason["escalated"].(float64) != 1 {
t.Fatalf("expected 1 escalated block, got %v", blockedByReason)
}
if out["twin_sends_blocked"].(float64) != 1 {
t.Fatalf("expected top-line twin_sends_blocked to match the sum of by-reason counts, got %v", out["twin_sends_blocked"])
}
}
// TestAdminMetricsBlockedByReasonAttributesDistinctCauses exercises two
// different twin-send block paths (peer veto, autonomy L0) through the real
// HTTP handler and checks /admin/metrics attributes each to its own reason
// key instead of collapsing them into one undifferentiated total.
func TestAdminMetricsBlockedByReasonAttributesDistinctCauses(t *testing.T) {
runtimeMetrics = &RuntimeMetrics{} // see comment in TestAdminMetricsCountsMessagesEscalationsAndVeto
server, db := setupTestServer(t)
senderID, _ := mustSignup(t, server.URL, "차단사유") // defaults to AutonomyL0
vetoedConv := Conversation{IsGroup: false}
if err := db.Create(&vetoedConv).Error; err != nil {
t.Fatalf("create conversation: %v", err)
}
vetoedBase := server.URL + "/conversations/" + strconv.FormatUint(uint64(vetoedConv.ID), 10)
postJSON(t, vetoedBase+"/veto", nil)
if resp := postJSON(t, vetoedBase+"/messages", sendMessageRequest{SenderID: senderID, Text: "안녕", SenderMode: SenderTwin}); resp.StatusCode != http.StatusForbidden {
t.Fatalf("expected peer-veto block, got %d", resp.StatusCode)
}
l0Conv := Conversation{IsGroup: false}
if err := db.Create(&l0Conv).Error; err != nil {
t.Fatalf("create conversation: %v", err)
}
l0Base := server.URL + "/conversations/" + strconv.FormatUint(uint64(l0Conv.ID), 10)
if resp := postJSON(t, l0Base+"/messages", sendMessageRequest{SenderID: senderID, Text: "안녕", SenderMode: SenderTwin}); resp.StatusCode != http.StatusForbidden {
t.Fatalf("expected L0 autonomy block, got %d", resp.StatusCode)
}
mreq, _ := http.NewRequest(http.MethodGet, server.URL+"/admin/metrics", nil)
mreq.Header.Set("Authorization", "Bearer test-admin-token")
resp, err := http.DefaultClient.Do(mreq)
if err != nil {
t.Fatalf("get metrics: %v", err)
}
var out map[string]interface{}
json.NewDecoder(resp.Body).Decode(&out)
blockedByReason := out["twin_sends_blocked_by_reason"].(map[string]interface{})
if blockedByReason["peer_veto"].(float64) != 1 {
t.Fatalf(`expected twin_sends_blocked_by_reason["peer_veto"] == 1, got %v`, blockedByReason)
}
if blockedByReason["autonomy_l0"].(float64) != 1 {
t.Fatalf(`expected twin_sends_blocked_by_reason["autonomy_l0"] == 1, got %v`, blockedByReason)
}
if out["twin_sends_blocked"].(float64) != 2 {
t.Fatalf("expected top-line twin_sends_blocked == 2, got %v", out["twin_sends_blocked"])
}
}
func TestWhitelistRuleCRUD(t *testing.T) {

View File

@ -17,6 +17,13 @@ type RuntimeMetrics struct {
EscalateChecks int64
EscalateErrors int64
TwinSendsBlocked int64
// TwinSendsBlockedByReason breaks the total above down by *why* a
// twin-authored send was blocked (peer_veto, flood_blocked,
// autonomy_l0, ...) -- see the call sites in main.go's
// POST /conversations/:id/messages handler for the full set of keys.
// This lets an operator tell a healthy signal (escalations catching
// sensitive content) apart from a bad one (lots of peer_veto/flood).
TwinSendsBlockedByReason map[string]int64
PushAttempts int64
PushSkipped int64
PushDelivered int64
@ -48,10 +55,14 @@ func (m *RuntimeMetrics) recordEscalate(err error) {
}
}
func (m *RuntimeMetrics) recordTwinBlocked() {
func (m *RuntimeMetrics) recordTwinBlocked(reason string) {
m.mu.Lock()
defer m.mu.Unlock()
m.TwinSendsBlocked++
if m.TwinSendsBlockedByReason == nil {
m.TwinSendsBlockedByReason = map[string]int64{}
}
m.TwinSendsBlockedByReason[reason]++
}
func (m *RuntimeMetrics) recordPush(delivered int, skipped bool) {
@ -88,6 +99,10 @@ func (m *RuntimeMetrics) snapshot() ginHMetrics {
if m.EscalateChecks > 0 {
escErrRate = float64(m.EscalateErrors) / float64(m.EscalateChecks)
}
byReason := make(map[string]int64, len(m.TwinSendsBlockedByReason))
for k, v := range m.TwinSendsBlockedByReason {
byReason[k] = v
}
return ginHMetrics{
DraftRequests: m.DraftRequests,
DraftErrors: m.DraftErrors,
@ -99,6 +114,7 @@ func (m *RuntimeMetrics) snapshot() ginHMetrics {
EscalateErrors: m.EscalateErrors,
EscalateErrorRate: escErrRate,
TwinSendsBlocked: m.TwinSendsBlocked,
TwinSendsBlockedByReason: byReason,
PushAttempts: m.PushAttempts,
PushSkipped: m.PushSkipped,
PushDelivered: m.PushDelivered,
@ -116,6 +132,7 @@ type ginHMetrics struct {
EscalateErrors int64
EscalateErrorRate float64
TwinSendsBlocked int64
TwinSendsBlockedByReason map[string]int64
PushAttempts int64
PushSkipped int64
PushDelivered int64

View File

@ -0,0 +1,45 @@
package main
import "testing"
// TestRecordTwinBlockedByReason confirms recordTwinBlocked attributes counts
// to the right reason key and still keeps a top-line total in sync, so
// existing dashboards/scripts reading twin_sends_blocked keep working
// alongside the new by-reason breakdown.
func TestRecordTwinBlockedByReason(t *testing.T) {
m := &RuntimeMetrics{}
m.recordTwinBlocked("x")
m.recordTwinBlocked("x")
m.recordTwinBlocked("y")
snap := m.snapshot()
if snap.TwinSendsBlocked != 3 {
t.Fatalf("expected top-line total 3, got %d", snap.TwinSendsBlocked)
}
if got := snap.TwinSendsBlockedByReason["x"]; got != 2 {
t.Fatalf(`expected TwinSendsBlockedByReason["x"] == 2, got %d`, got)
}
if got := snap.TwinSendsBlockedByReason["y"]; got != 1 {
t.Fatalf(`expected TwinSendsBlockedByReason["y"] == 1, got %d`, got)
}
if len(snap.TwinSendsBlockedByReason) != 2 {
t.Fatalf("expected exactly 2 distinct reasons, got %v", snap.TwinSendsBlockedByReason)
}
}
// TestRecordTwinBlockedSnapshotIsolation confirms the snapshot's map is a
// copy, not an alias into the live struct -- so a caller mutating the
// returned map (e.g. JSON-encoding it, or a test asserting on it) can't
// corrupt runtimeMetrics's internal state.
func TestRecordTwinBlockedSnapshotIsolation(t *testing.T) {
m := &RuntimeMetrics{}
m.recordTwinBlocked("peer_veto")
snap := m.snapshot()
snap.TwinSendsBlockedByReason["peer_veto"] = 999
again := m.snapshot()
if again.TwinSendsBlockedByReason["peer_veto"] != 1 {
t.Fatalf("snapshot map leaked a live reference into RuntimeMetrics, got %v", again.TwinSendsBlockedByReason)
}
}

View File

@ -121,11 +121,21 @@
`/admin/metrics``peer_veto_rate`로 1차 근사 가능해짐(대화방 단위, 확정 정의 아님). 자연스러움
피드백 수집 UI는 Flutter 클라이언트 책임이라 보류. 안전선 위반 0건은 런타임에 "수집"하는 지표라기
보다 지금까지의 하드게이트 테스트들이 이미 보증하는 것 — 별도 계측 불필요
- [~] 모니터링 대시보드 (에스컬레이션 트리거율, 생성 지연시간, 오류율) — `GET /admin/metrics`
카운트 기반 데이터(메시지 수, 에스컬레이션 사유별 집계, 거부권 발동률, 초대 코드 발급/사용 수)는
노출함. **대시보드 UI 자체와 생성 지연시간·오류율**은 아직 없음 — UI는 Flutter/관리자 웹 쪽이고,
지연시간·오류율은 요청 타이밍/로깅 계측 계층이 따로 필요해서 이번엔 만들지 않음(허위로 채우지
않고 명시적으로 비워둠)
- [x] 모니터링 대시보드 (에스컬레이션 트리거율, 생성 지연시간, 오류율) — **2026-08-03 재확인**:
이 항목의 `[~]` 상태와 "대시보드 UI 자체와 생성 지연시간·오류율은 아직 없음" 설명은 실제로는
낡은 기록이었음(§B의 "생성 지연시간·오류율 계측"/"모니터링 대시보드(최소)" `[x]` 항목과 서로
모순되고 있었던 걸 오늘 발견). 실제로는 이전 Phase 1 B 커밋에서 이미 `GET /admin/dashboard`
(HTML, `core-backend/b_routes.go``adminDashboardHTML`)와 그 위에서 읽는
`GET /admin/metrics`(`core-backend/main.go`, 카드: 사용자·메시지·에스컬레이션·거부권 발동률·
생성 지연시간 평균/최대·에스컬레이션 오류율 등)가 구현되어 있었음. 오늘 실제로 추가한 것은
두 가지뿐: (1) 트윈 발송 차단을 사유별로 분해하는 `twin_sends_blocked_by_reason`
(peer_veto/group_conversation/flood_blocked/flood_detected/escalate_check_error/escalated/
autonomy_l0/autonomy_l1_unapproved/autonomy_l2_no_whitelist_match/autonomy_unknown_level —
이전엔 전부 `twin_sends_blocked` 한 총합으로만 뭉쳐 있었음), (2) 이미 JSON에는 있었지만
대시보드 화면에는 안 보이던 `escalations_by_reason`과 새 `twin_sends_blocked_by_reason`
대시보드에 사유별 표로 렌더링. 정직하게 남겨둘 한계: 이 계측은 여전히 프로세스 메모리
전용(`RuntimeMetrics`)이라 재시작하면 리셋되는 근사치이고, 진짜 타임시리즈 DB가 아님 —
트렌드 분석이 필요해지면 별도 작업으로 남음
**2.7 콘텐츠 갭 — PRD 3.1 P0 대비 미구현 기능** (2026-07-31 발견, 2026-08-03 2차 재분석으로
2.7-D~F 추가)