jeet/grateful-journal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added encryption

Browse Source
This commit is contained in:
Jeet Debnath
2026-03-09 10:54:07 +05:30
parent 6e184dc590
commit 6720e28d08
27 changed files with 2093 additions and 709 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
backend/MIGRATION_GUIDE.md
View File

@@ -92,6 +92,7 @@ python scripts/migrate_data.py
**Script Output:**
The script will:
1. Report duplicate users found
2. Map old duplicate user IDs to the canonical (oldest) user
3. Update all entries to reference the canonical user
@@ -292,6 +293,7 @@ npm run dev # or your dev command
```
Test the full application:
- Login via Google
- Create an entry
- View entries in history
@@ -320,6 +322,7 @@ This will revert the database to its pre-migration state.
**Cause:** Some entries still have string userId references.
**Fix:** Re-run the migration script:
```bash
python backend/scripts/migrate_data.py
```
@@ -328,6 +331,7 @@ python backend/scripts/migrate_data.py
**Cause:** userId is still a string in old entries.
**Fix:** Check the entry structure:
```bash
mongosh --db grateful_journal
db.entries.findOne() # Check userId type
@@ -339,6 +343,7 @@ If userId is a string, run migration again.
**Cause:** Index creation failed due to duplicate emails.
**Fix:** The migration script handles this, but if you hit this:
```bash
# Rerun migration
python scripts/migrate_data.py

113
backend/REFACTORING_SUMMARY.md
View File

@@ -30,56 +30,56 @@ This refactoring addresses critical database issues and optimizes the MongoDB sc
### Backend Core
1. **[models.py](./models.py)** — Updated Pydantic models
- Changed `User.id: str` → now uses `_id` alias for ObjectId
- Added `JournalEntry.entryDate: datetime`
- Added `EncryptionMetadata` model for encryption support
- Added pagination response models
- Changed `User.id: str` → now uses `_id` alias for ObjectId
- Added `JournalEntry.entryDate: datetime`
- Added `EncryptionMetadata` model for encryption support
- Added pagination response models
2. **[routers/users.py](./routers/users.py)** — Rewrote user logic
- Changed user registration from `insert_one` → `update_one` with upsert
- Prevents duplicate users (one per email)
- Validates ObjectId conversions with error handling
- Added `get_user_by_id` endpoint
- Changed user registration from `insert_one` → `update_one` with upsert
- Prevents duplicate users (one per email)
- Validates ObjectId conversions with error handling
- Added `get_user_by_id` endpoint
3. **[routers/entries.py](./routers/entries.py)** — Updated entry handling
- Convert all `userId` from string → ObjectId
- Enforce user existence check before entry creation
- Added `entryDate` field support
- Added `get_entries_by_month` for calendar queries
- Improved pagination with `hasMore` flag
- Better error messages for invalid ObjectIds
- Convert all `userId` from string → ObjectId
- Enforce user existence check before entry creation
- Added `entryDate` field support
- Added `get_entries_by_month` for calendar queries
- Improved pagination with `hasMore` flag
- Better error messages for invalid ObjectIds
### New Scripts
4. **[scripts/migrate_data.py](./scripts/migrate_data.py)** — Data migration
- Deduplicates users by email (keeps oldest)
- Converts `entries.userId` string → ObjectId
- Adds `entryDate` field (defaults to createdAt)
- Adds encryption metadata
- Verifies data integrity post-migration
- Deduplicates users by email (keeps oldest)
- Converts `entries.userId` string → ObjectId
- Adds `entryDate` field (defaults to createdAt)
- Adds encryption metadata
- Verifies data integrity post-migration
5. **[scripts/create_indexes.py](./scripts/create_indexes.py)** — Index creation
- Creates unique index on `users.email`
- Creates compound indexes:
- `entries(userId, createdAt)` — for history/pagination
- `entries(userId, entryDate)` — for calendar view
- Creates supporting indexes for tags and dates
- Creates unique index on `users.email`
- Creates compound indexes:
- `entries(userId, createdAt)` — for history/pagination
- `entries(userId, entryDate)` — for calendar view
- Creates supporting indexes for tags and dates
### Documentation
6. **[SCHEMA.md](./SCHEMA.md)** — Complete schema documentation
- Full field descriptions and examples
- Index rationale and usage
- Query patterns with examples
- Data type conversions
- Security considerations
- Full field descriptions and examples
- Index rationale and usage
- Query patterns with examples
- Data type conversions
- Security considerations
7. **[MIGRATION_GUIDE.md](./MIGRATION_GUIDE.md)** — Step-by-step migration
- Pre-migration checklist
- Backup instructions
- Running migration and index scripts
- Rollback procedure
- Troubleshooting guide
- Pre-migration checklist
- Backup instructions
- Running migration and index scripts
- Rollback procedure
- Troubleshooting guide
---
@@ -100,6 +100,7 @@ This refactoring addresses critical database issues and optimizes the MongoDB sc
```
**Key Changes:**
- ✓ Unique email index
- ✓ Settings embedded (theme field)
- ✓ No separate settings collection
@@ -115,11 +116,11 @@ This refactoring addresses critical database issues and optimizes the MongoDB sc
mood: string | null,
tags: string[],
isPublic: boolean,
entryDate: datetime, // ← NEW: Logical journal date
createdAt: datetime,
updatedAt: datetime,
encryption: { // ← NEW: Encryption metadata
encrypted: boolean,
iv: string | null,
@@ -129,6 +130,7 @@ This refactoring addresses critical database issues and optimizes the MongoDB sc
```
**Key Changes:**
- ✓ `userId` is ObjectId
- ✓ `entryDate` separates "when written" (createdAt) from "which day it's for" (entryDate)
- ✓ Encryption metadata for future encrypted storage
@@ -141,12 +143,14 @@ This refactoring addresses critical database issues and optimizes the MongoDB sc
### User Registration (Upsert)
**Old:**
```python
POST /api/users/register
# Created new user every time (duplicates!)
```
**New:**
```python
POST /api/users/register
# Idempotent: updates if exists, inserts if not
@@ -156,6 +160,7 @@ POST /api/users/register
### Get User by ID
**New Endpoint:**
```
GET /api/users/{user_id}
```
@@ -165,6 +170,7 @@ Returns user by ObjectId instead of only by email.
### Create Entry
**Old:**
```json
POST /api/entries/{user_id}
{
@@ -174,6 +180,7 @@ POST /api/entries/{user_id}
```
**New:**
```json
POST /api/entries/{user_id}
{
@@ -191,6 +198,7 @@ POST /api/entries/{user_id}
### Get Entries
**Improved Response:**
```json
{
"entries": [...],
@@ -206,6 +214,7 @@ POST /api/entries/{user_id}
### New Endpoint: Get Entries by Month
**For Calendar View:**
```
GET /api/entries/{user_id}/by-month/{year}/{month}?limit=100
```
@@ -314,6 +323,7 @@ No breaking changes if using the API correctly. However:
### Backup Created
✓ Before migration, create backup:
```bash
mongodump --db grateful_journal --out ./backup-2026-03-05
```
@@ -321,6 +331,7 @@ mongodump --db grateful_journal --out ./backup-2026-03-05
### Rollback Available
If issues occur:
```bash
mongorestore --drop --db grateful_journal ./backup-2026-03-05
```
@@ -396,26 +407,28 @@ Based on this new schema, future features are now possible:
If you encounter issues during or after migration:
1. **Check logs:**
```bash
tail -f backend/logs/backend.log
```
1. **Check logs:**
```bash
tail -f backend/logs/backend.log
```
2. **Verify database:**
```bash
mongosh --db grateful_journal
db.users.countDocuments({})
db.entries.countDocuments({})
```
```bash
mongosh --db grateful_journal
db.users.countDocuments({})
db.entries.countDocuments({})
```
3. **Review documents:**
- [SCHEMA.md](./SCHEMA.md) — Schema reference
- [MIGRATION_GUIDE.md](./MIGRATION_GUIDE.md) — Troubleshooting section
- [models.py](./models.py) — Pydantic model definitions
- [SCHEMA.md](./SCHEMA.md) — Schema reference
- [MIGRATION_GUIDE.md](./MIGRATION_GUIDE.md) — Troubleshooting section
- [models.py](./models.py) — Pydantic model definitions
4. **Consult code:**
- [routers/users.py](./routers/users.py) — User logic
- [routers/entries.py](./routers/entries.py) — Entry logic
- [routers/users.py](./routers/users.py) — User logic
- [routers/entries.py](./routers/entries.py) — Entry logic
---

86
backend/SCHEMA.md
View File

@@ -39,15 +39,15 @@ Stores user profile information. One document per unique email.
#### Field Descriptions
| Field | Type | Required | Notes |
| ----------- | ------ | -------- | ----------------------------------------- |
| `_id` | ObjectId | Yes | Unique primary key, auto-generated |
| `email` | String | Yes | User's email; unique constraint; indexed |
| `displayName` | String | Yes | User's display name (from Google Auth) |
| `photoURL` | String | No | User's profile photo URL |
| `theme` | String | Yes | Theme preference: "light" or "dark" |
| `createdAt` | Date | Yes | Account creation timestamp |
| `updatedAt` | Date | Yes | Last profile update timestamp |
| Field | Type | Required | Notes |
| ------------- | -------- | -------- | ---------------------------------------- |
| `_id` | ObjectId | Yes | Unique primary key, auto-generated |
| `email` | String | Yes | User's email; unique constraint; indexed |
| `displayName` | String | Yes | User's display name (from Google Auth) |
| `photoURL` | String | No | User's profile photo URL |
| `theme` | String | Yes | Theme preference: "light" or "dark" |
| `createdAt` | Date | Yes | Account creation timestamp |
| `updatedAt` | Date | Yes | Last profile update timestamp |
#### Unique Constraints
@@ -84,11 +84,11 @@ Stores journal entries for each user. Each entry has a logical journal date and
mood: "happy" | "sad" | "neutral" | "anxious" | "grateful" | null,
tags: string[],
isPublic: boolean,
entryDate: Date, // Logical journal date
createdAt: Date,
updatedAt: Date,
encryption: {
encrypted: boolean,
iv: string | null, // Base64-encoded initialization vector
@@ -99,19 +99,19 @@ Stores journal entries for each user. Each entry has a logical journal date and
#### Field Descriptions
| Field | Type | Required | Notes |
| ---------- | ------ | -------- | ------------------------------------------ |
| `_id` | ObjectId | Yes | Entry ID; auto-generated; indexed |
| `userId` | ObjectId | Yes | Reference to user._id; indexed; enforced |
| `title` | String | Yes | Entry title/headline |
| `content` | String | Yes | Entry body content |
| `mood` | String | No | Mood selector (null if not set) |
| `tags` | Array | Yes | Array of user-defined tags [] |
| `isPublic` | Bool | Yes | Public sharing flag (currently unused) |
| `entryDate` | Date | Yes | Logical journal date (start of day, UTC) |
| `createdAt` | Date | Yes | Database write timestamp |
| `updatedAt` | Date | Yes | Last modification timestamp |
| `encryption` | Object | Yes | Encryption metadata (nested) |
| Field | Type | Required | Notes |
| ------------ | -------- | -------- | ----------------------------------------- |
| `_id` | ObjectId | Yes | Entry ID; auto-generated; indexed |
| `userId` | ObjectId | Yes | Reference to user.\_id; indexed; enforced |
| `title` | String | Yes | Entry title/headline |
| `content` | String | Yes | Entry body content |
| `mood` | String | No | Mood selector (null if not set) |
| `tags` | Array | Yes | Array of user-defined tags [] |
| `isPublic` | Bool | Yes | Public sharing flag (currently unused) |
| `entryDate` | Date | Yes | Logical journal date (start of day, UTC) |
| `createdAt` | Date | Yes | Database write timestamp |
| `updatedAt` | Date | Yes | Last modification timestamp |
| `encryption` | Object | Yes | Encryption metadata (nested) |
#### Encryption Metadata
@@ -124,6 +124,7 @@ Stores journal entries for each user. Each entry has a logical journal date and
```
**Notes:**
- `encrypted: false` by default (plain text storage)
- When setting `encrypted: true`, client provides `iv` and `algorithm`
- Server stores metadata but does NOT decrypt; decryption happens client-side
@@ -160,26 +161,26 @@ Indexes optimize query performance. All indexes are created by the `scripts/crea
```javascript
// Unique index on email (prevents duplicates)
db.users.createIndex({ email: 1 }, { unique: true })
db.users.createIndex({ email: 1 }, { unique: true });
// For sorting users by creation date
db.users.createIndex({ createdAt: -1 })
db.users.createIndex({ createdAt: -1 });
```
### Entries Indexes
```javascript
// Compound index for history pagination (most recent first)
db.entries.createIndex({ userId: 1, createdAt: -1 })
db.entries.createIndex({ userId: 1, createdAt: -1 });
// Compound index for calendar queries by date
db.entries.createIndex({ userId: 1, entryDate: 1 })
db.entries.createIndex({ userId: 1, entryDate: 1 });
// For tag-based searches (future feature)
db.entries.createIndex({ tags: 1 })
db.entries.createIndex({ tags: 1 });
// For sorting by entry date
db.entries.createIndex({ entryDate: -1 })
db.entries.createIndex({ entryDate: -1 });
```
### Index Rationale
@@ -385,15 +386,15 @@ iso_string = dt.isoformat()
### What Changed
| Aspect | Old Schema | New Schema |
| -------------- | ------------------------- | ------------------------------- |
| Users | Many per email possible | One per email (unique) |
| User _id | ObjectId (correct) | ObjectId (unchanged) |
| Entry userId | String | ObjectId |
| Entry date | Only `createdAt` | `createdAt` + `entryDate` |
| Encryption | Not supported | Metadata in `encryption` field |
| Settings | Separate collection | Merged into `users.theme` |
| Indexes | None | Comprehensive indexes |
| Aspect | Old Schema | New Schema |
| ------------ | ----------------------- | ------------------------------ |
| Users | Many per email possible | One per email (unique) |
| User \_id | ObjectId (correct) | ObjectId (unchanged) |
| Entry userId | String | ObjectId |
| Entry date | Only `createdAt` | `createdAt` + `entryDate` |
| Encryption | Not supported | Metadata in `encryption` field |
| Settings | Separate collection | Merged into `users.theme` |
| Indexes | None | Comprehensive indexes |
### Migration Steps
@@ -471,7 +472,8 @@ mongorestore --db grateful_journal ./backup-entries
### Q: How do I encrypt entry content?
**A:**
**A:**
1. Client encrypts content client-side using a key (not transmitted)
2. Client sends encrypted content + metadata (iv, algorithm)
3. Server stores content + encryption metadata as-is
@@ -480,14 +482,17 @@ mongorestore --db grateful_journal ./backup-entries
### Q: What if I have duplicate users?
**A:** Run the migration script:
```bash
python backend/scripts/migrate_data.py
```
It detects duplicates, keeps the oldest, and consolidates entries.
### Q: Should I paginate entries?
**A:** Yes. Use `skip` and `limit` to prevent loading thousands of entries:
```
GET /api/entries/{user_id}?skip=0&limit=50
```
@@ -495,6 +500,7 @@ GET /api/entries/{user_id}?skip=0&limit=50
### Q: How do I query entries by date range?
**A:** Use the calendar endpoint or build a query:
```python
db.entries.find({
"userId": oid,

BIN
backend/__pycache__/models.cpython-312.pyc
View File

Binary file not shown.

55
backend/models.py
View File

@@ -1,4 +1,4 @@
from pydantic import BaseModel, Field # type: ignore
from pydantic import BaseModel, Field # type: ignore
from datetime import datetime
from typing import Optional, List
from enum import Enum
@@ -85,35 +85,43 @@ class MoodEnum(str, Enum):
class EncryptionMetadata(BaseModel):
"""Optional encryption metadata for entries"""
encrypted: bool = False
iv: Optional[str] = None # Initialization vector as base64 string
algorithm: Optional[str] = None # e.g., "AES-256-GCM"
"""Encryption metadata for entries - zero-knowledge privacy"""
encrypted: bool = True
ciphertext: str # Base64-encoded encrypted content
nonce: str # Base64-encoded nonce used for encryption
algorithm: str = "XSalsa20-Poly1305" # crypto_secretbox algorithm
class Config:
json_schema_extra = {
"example": {
"encrypted": False,
"iv": None,
"algorithm": None
"encrypted": True,
"ciphertext": "base64_encoded_ciphertext...",
"nonce": "base64_encoded_nonce...",
"algorithm": "XSalsa20-Poly1305"
}
}
class JournalEntryCreate(BaseModel):
title: str
content: str
title: Optional[str] = None # Optional if encrypted
content: Optional[str] = None # Optional if encrypted
mood: Optional[MoodEnum] = None
tags: Optional[List[str]] = None
isPublic: Optional[bool] = False
entryDate: Optional[datetime] = None # Logical journal date; defaults to today
# Logical journal date; defaults to today
entryDate: Optional[datetime] = None
# Encryption metadata - present if entry is encrypted
encryption: Optional[EncryptionMetadata] = None
class Config:
json_schema_extra = {
"example": {
"title": "Today's Gratitude",
"content": "I'm grateful for...",
"encryption": {
"encrypted": True,
"ciphertext": "base64_ciphertext...",
"nonce": "base64_nonce...",
"algorithm": "XSalsa20-Poly1305"
},
"mood": "grateful",
"tags": ["work", "family"],
"isPublic": False,
@@ -142,15 +150,15 @@ class JournalEntryUpdate(BaseModel):
class JournalEntry(BaseModel):
id: str = Field(alias="_id")
userId: str # ObjectId as string
title: str
content: str
title: Optional[str] = None # None if encrypted
content: Optional[str] = None # None if encrypted
mood: Optional[MoodEnum] = None
tags: Optional[List[str]] = []
isPublic: bool = False
entryDate: datetime # Logical journal date
createdAt: datetime
updatedAt: datetime
encryption: EncryptionMetadata = Field(default_factory=lambda: EncryptionMetadata())
encryption: Optional[EncryptionMetadata] = None # Present if encrypted
class Config:
from_attributes = True
@@ -159,19 +167,18 @@ class JournalEntry(BaseModel):
"example": {
"_id": "507f1f77bcf86cd799439011",
"userId": "507f1f77bcf86cd799439012",
"title": "Today's Gratitude",
"content": "I'm grateful for...",
"encryption": {
"encrypted": True,
"ciphertext": "base64_ciphertext...",
"nonce": "base64_nonce...",
"algorithm": "XSalsa20-Poly1305"
},
"mood": "grateful",
"tags": ["work", "family"],
"isPublic": False,
"entryDate": "2026-03-05T00:00:00Z",
"createdAt": "2026-03-05T12:00:00Z",
"updatedAt": "2026-03-05T12:00:00Z",
"encryption": {
"encrypted": False,
"iv": None,
"algorithm": None
}
"updatedAt": "2026-03-05T12:00:00Z"
}
}

BIN
backend/routers/__pycache__/entries.cpython-312.pyc
View File

Binary file not shown.

BIN
backend/routers/__pycache__/users.cpython-312.pyc
View File

Binary file not shown.

108
backend/routers/entries.py
View File

@@ -15,19 +15,16 @@ def _format_entry(entry: dict) -> dict:
return {
"id": str(entry["_id"]),
"userId": str(entry["userId"]),
"title": entry.get("title", ""),
"content": entry.get("content", ""),
"title": entry.get("title"), # None if encrypted
"content": entry.get("content"), # None if encrypted
"mood": entry.get("mood"),
"tags": entry.get("tags", []),
"isPublic": entry.get("isPublic", False),
"entryDate": entry.get("entryDate", entry.get("createdAt")).isoformat() if entry.get("entryDate") or entry.get("createdAt") else None,
"createdAt": entry["createdAt"].isoformat(),
"updatedAt": entry["updatedAt"].isoformat(),
"encryption": entry.get("encryption", {
"encrypted": False,
"iv": None,
"algorithm": None
})
# Full encryption metadata including ciphertext and nonce
"encryption": entry.get("encryption")
}
@@ -35,51 +32,70 @@ def _format_entry(entry: dict) -> dict:
async def create_entry(user_id: str, entry_data: JournalEntryCreate):
"""
Create a new journal entry.
For encrypted entries:
- Send encryption metadata with ciphertext and nonce
- Omit title and content (they're encrypted in ciphertext)
For unencrypted entries (deprecated):
- Send title and content directly
entryDate: The logical journal date for this entry (defaults to today UTC).
createdAt: Database write timestamp.
Server stores only: encrypted ciphertext, nonce, and metadata.
Server never sees plaintext.
"""
db = get_database()
try:
user_oid = ObjectId(user_id)
# Verify user exists
user = db.users.find_one({"_id": user_oid})
if not user:
raise HTTPException(status_code=404, detail="User not found")
now = datetime.utcnow()
entry_date = entry_data.entryDate or now.replace(hour=0, minute=0, second=0, microsecond=0)
entry_date = entry_data.entryDate or now.replace(
hour=0, minute=0, second=0, microsecond=0)
# Validate encryption metadata if present
if entry_data.encryption:
if not entry_data.encryption.ciphertext or not entry_data.encryption.nonce:
raise HTTPException(
status_code=400,
detail="Encryption metadata must include ciphertext and nonce"
)
entry_doc = {
"userId": user_oid,
"title": entry_data.title,
"content": entry_data.content,
"title": entry_data.title, # None if encrypted
"content": entry_data.content, # None if encrypted
"mood": entry_data.mood,
"tags": entry_data.tags or [],
"isPublic": entry_data.isPublic or False,
"entryDate": entry_date, # Logical journal date
"createdAt": now,
"updatedAt": now,
"encryption": entry_data.encryption.model_dump() if entry_data.encryption else {
"encrypted": False,
"iv": None,
"algorithm": None
}
"encryption": entry_data.encryption.model_dump() if entry_data.encryption else None
}
result = db.entries.insert_one(entry_doc)
return {
"id": str(result.inserted_id),
"userId": user_id,
"message": "Entry created successfully"
}
except HTTPException:
raise
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid user ID format")
raise HTTPException(status_code=500, detail=f"Failed to create entry: {str(e)}")
raise HTTPException(
status_code=400, detail="Invalid user ID format")
raise HTTPException(
status_code=500, detail=f"Failed to create entry: {str(e)}")
@router.get("/{user_id}")
@@ -90,14 +106,14 @@ async def get_user_entries(
):
"""
Get paginated entries for a user (most recent first).
Supports pagination via skip and limit.
"""
db = get_database()
try:
user_oid = ObjectId(user_id)
# Verify user exists
user = db.users.find_one({"_id": user_oid})
if not user:
@@ -112,7 +128,7 @@ async def get_user_entries(
# Format entries
formatted_entries = [_format_entry(entry) for entry in entries]
# Get total count
total = db.entries.count_documents({"userId": user_oid})
has_more = (skip + limit) < total
@@ -128,8 +144,10 @@ async def get_user_entries(
}
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid user ID format")
raise HTTPException(status_code=500, detail=f"Failed to fetch entries: {str(e)}")
raise HTTPException(
status_code=400, detail="Invalid user ID format")
raise HTTPException(
status_code=500, detail=f"Failed to fetch entries: {str(e)}")
@router.get("/{user_id}/{entry_id}")
@@ -153,7 +171,8 @@ async def get_entry(user_id: str, entry_id: str):
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid ID format")
raise HTTPException(status_code=500, detail=f"Failed to fetch entry: {str(e)}")
raise HTTPException(
status_code=500, detail=f"Failed to fetch entry: {str(e)}")
@router.put("/{user_id}/{entry_id}")
@@ -170,7 +189,8 @@ async def update_entry(user_id: str, entry_id: str, entry_data: JournalEntryUpda
# If entryDate provided in update data, ensure it's a datetime
if "entryDate" in update_data and isinstance(update_data["entryDate"], str):
update_data["entryDate"] = datetime.fromisoformat(update_data["entryDate"].replace("Z", "+00:00"))
update_data["entryDate"] = datetime.fromisoformat(
update_data["entryDate"].replace("Z", "+00:00"))
result = db.entries.update_one(
{
@@ -189,7 +209,8 @@ async def update_entry(user_id: str, entry_id: str, entry_data: JournalEntryUpda
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid ID format")
raise HTTPException(status_code=500, detail=f"Failed to update entry: {str(e)}")
raise HTTPException(
status_code=500, detail=f"Failed to update entry: {str(e)}")
@router.delete("/{user_id}/{entry_id}")
@@ -213,21 +234,22 @@ async def delete_entry(user_id: str, entry_id: str):
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid ID format")
raise HTTPException(status_code=500, detail=f"Failed to delete entry: {str(e)}")
raise HTTPException(
status_code=500, detail=f"Failed to delete entry: {str(e)}")
@router.get("/{user_id}/by-date/{date_str}")
async def get_entries_by_date(user_id: str, date_str: str):
"""
Get entries for a specific date (format: YYYY-MM-DD).
Matches entries by entryDate field.
"""
db = get_database()
try:
user_oid = ObjectId(user_id)
# Parse date
target_date = datetime.strptime(date_str, "%Y-%m-%d")
next_date = target_date + timedelta(days=1)
@@ -254,25 +276,28 @@ async def get_entries_by_date(user_id: str, date_str: str):
status_code=400, detail="Invalid date format. Use YYYY-MM-DD")
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid user ID format")
raise HTTPException(status_code=500, detail=f"Failed to fetch entries: {str(e)}")
raise HTTPException(
status_code=400, detail="Invalid user ID format")
raise HTTPException(
status_code=500, detail=f"Failed to fetch entries: {str(e)}")
@router.get("/{user_id}/by-month/{year}/{month}")
async def get_entries_by_month(user_id: str, year: int, month: int, limit: int = Query(100, ge=1, le=500)):
"""
Get entries for a specific month (for calendar view).
Query format: GET /api/entries/{user_id}/by-month/{year}/{month}?limit=100
"""
db = get_database()
try:
user_oid = ObjectId(user_id)
if not (1 <= month <= 12):
raise HTTPException(status_code=400, detail="Month must be between 1 and 12")
raise HTTPException(
status_code=400, detail="Month must be between 1 and 12")
# Calculate date range
start_date = datetime(year, month, 1)
if month == 12:
@@ -302,8 +327,10 @@ async def get_entries_by_month(user_id: str, year: int, month: int, limit: int =
raise HTTPException(status_code=400, detail="Invalid year or month")
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid user ID format")
raise HTTPException(status_code=500, detail=f"Failed to fetch entries: {str(e)}")
raise HTTPException(
status_code=400, detail="Invalid user ID format")
raise HTTPException(
status_code=500, detail=f"Failed to fetch entries: {str(e)}")
@router.post("/convert-timestamp/utc-to-ist")
@@ -323,4 +350,5 @@ async def convert_utc_to_ist(data: dict):
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=f"Conversion failed: {str(e)}")
raise HTTPException(
status_code=500, detail=f"Conversion failed: {str(e)}")

26
backend/routers/users.py
View File

@@ -14,7 +14,7 @@ router = APIRouter()
async def register_user(user_data: UserCreate):
"""
Register or get user (idempotent).
Uses upsert pattern to ensure one user per email.
If user already exists, returns existing user.
Called after Firebase Google Auth on frontend.
@@ -43,7 +43,8 @@ async def register_user(user_data: UserCreate):
# Fetch the user (either newly created or existing)
user = db.users.find_one({"email": user_data.email})
if not user:
raise HTTPException(status_code=500, detail="Failed to retrieve user after upsert")
raise HTTPException(
status_code=500, detail="Failed to retrieve user after upsert")
return {
"id": str(user["_id"]),
@@ -56,7 +57,8 @@ async def register_user(user_data: UserCreate):
"message": "User registered successfully" if result.upserted_id else "User already exists"
}
except Exception as e:
raise HTTPException(status_code=500, detail=f"Registration failed: {str(e)}")
raise HTTPException(
status_code=500, detail=f"Registration failed: {str(e)}")
@router.get("/by-email/{email}", response_model=dict)
@@ -79,7 +81,8 @@ async def get_user_by_email(email: str):
"updatedAt": user["updatedAt"].isoformat()
}
except Exception as e:
raise HTTPException(status_code=500, detail=f"Failed to fetch user: {str(e)}")
raise HTTPException(
status_code=500, detail=f"Failed to fetch user: {str(e)}")
@router.get("/{user_id}", response_model=dict)
@@ -103,8 +106,10 @@ async def get_user_by_id(user_id: str):
}
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid user ID format")
raise HTTPException(status_code=500, detail=f"Failed to fetch user: {str(e)}")
raise HTTPException(
status_code=400, detail="Invalid user ID format")
raise HTTPException(
status_code=500, detail=f"Failed to fetch user: {str(e)}")
@router.put("/{user_id}", response_model=dict)
@@ -139,7 +144,8 @@ async def update_user(user_id: str, user_data: UserUpdate):
}
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid user ID format")
raise HTTPException(
status_code=400, detail="Invalid user ID format")
raise HTTPException(status_code=500, detail=f"Update failed: {str(e)}")
@@ -164,8 +170,10 @@ async def delete_user(user_id: str):
}
except Exception as e:
if "invalid ObjectId" in str(e).lower():
raise HTTPException(status_code=400, detail="Invalid user ID format")
raise HTTPException(status_code=500, detail=f"Deletion failed: {str(e)}")
raise HTTPException(
status_code=400, detail="Invalid user ID format")
raise HTTPException(
status_code=500, detail=f"Deletion failed: {str(e)}")
# Delete all entries by user
db.entries.delete_many({"userId": user_id})

32
backend/scripts/create_indexes.py
View File

@@ -15,18 +15,18 @@ from typing import Dict, List, Tuple
def create_indexes():
"""Create all required MongoDB indexes."""
settings = get_settings()
client = MongoClient(settings.mongodb_uri)
db = client[settings.mongodb_db_name]
print(f"✓ Connected to MongoDB: {settings.mongodb_db_name}\n")
indexes_created = []
# ========== USERS COLLECTION INDEXES ==========
print("Creating indexes for 'users' collection...")
# Unique index on email
try:
db.users.create_index(
@@ -38,7 +38,7 @@ def create_indexes():
print(" ✓ Created unique index on email")
except Exception as e:
print(f" ⚠ Email index: {e}")
# Index on createdAt for sorting
try:
db.users.create_index(
@@ -49,10 +49,10 @@ def create_indexes():
print(" ✓ Created index on createdAt")
except Exception as e:
print(f" ⚠ createdAt index: {e}")
# ========== ENTRIES COLLECTION INDEXES ==========
print("\nCreating indexes for 'entries' collection...")
# Compound index: userId + createdAt (for history pagination)
try:
db.entries.create_index(
@@ -63,7 +63,7 @@ def create_indexes():
print(" ✓ Created compound index on (userId, createdAt)")
except Exception as e:
print(f" ⚠ userId_createdAt index: {e}")
# Compound index: userId + entryDate (for calendar queries)
try:
db.entries.create_index(
@@ -74,7 +74,7 @@ def create_indexes():
print(" ✓ Created compound index on (userId, entryDate)")
except Exception as e:
print(f" ⚠ userId_entryDate index: {e}")
# Index on tags for searching (optional, for future)
try:
db.entries.create_index(
@@ -85,7 +85,7 @@ def create_indexes():
print(" ✓ Created index on tags")
except Exception as e:
print(f" ⚠ tags index: {e}")
# Index on entryDate range queries (for calendar)
try:
db.entries.create_index(
@@ -96,7 +96,7 @@ def create_indexes():
print(" ✓ Created index on entryDate")
except Exception as e:
print(f" ⚠ entryDate index: {e}")
# TTL Index on entries (optional: for auto-deleting old entries if needed)
# Uncomment if you want entries to auto-delete after 2 years
# try:
@@ -108,7 +108,7 @@ def create_indexes():
# print(" ✓ Created TTL index on createdAt (2 years)")
# except Exception as e:
# print(f" ⚠ TTL index: {e}")
# ========== SUMMARY ==========
print(f"\n{'='*60}")
print(f"✓ Index Creation Complete")
@@ -116,18 +116,18 @@ def create_indexes():
print(f"Total indexes created: {len(indexes_created)}")
for collection, index_name in indexes_created:
print(f" • {collection}.{index_name}")
# Optional: Print summary of all indexes
print(f"\n{'='*60}")
print("All Indexes Summary")
print(f"{'='*60}")
for collection_name in ["users", "entries"]:
print(f"\n{collection_name}:")
collection = db[collection_name]
for index_info in collection.list_indexes():
print(f" • {index_info['name']}")
client.close()
print("\n✓ Disconnected from MongoDB")

89
backend/scripts/migrate_data.py
View File

@@ -27,21 +27,21 @@ import sys
def migrate_data():
"""Perform complete data migration."""
settings = get_settings()
client = MongoClient(settings.mongodb_uri)
db = client[settings.mongodb_db_name]
print(f"✓ Connected to MongoDB: {settings.mongodb_db_name}\n")
# ========== STEP 1: DEDUPLICATE USERS ==========
print("=" * 70)
print("STEP 1: Deduplicating Users (keeping oldest)")
print("=" * 70)
duplicate_count = 0
user_mapping = {} # Maps old duplicates to canonical user ID
# Group users by email
email_groups = {}
for user in db.users.find():
@@ -49,7 +49,7 @@ def migrate_data():
if email not in email_groups:
email_groups[email] = []
email_groups[email].append(user)
# Process each email group
for email, users in email_groups.items():
if len(users) > 1:
@@ -57,52 +57,53 @@ def migrate_data():
users.sort(key=lambda u: u["createdAt"])
canonical_user = users[0]
canonical_id = canonical_user["_id"]
print(f"\n📧 Email: {email}")
print(f" Found {len(users)} duplicate users")
print(f" Keeping (earliest): {canonical_id}")
# Map all other users to canonical
for dup_user in users[1:]:
dup_id = dup_user["_id"]
user_mapping[str(dup_id)] = canonical_id
duplicate_count += 1
print(f" Deleting (later): {dup_id}")
# Delete duplicate users
for user in users[1:]:
db.users.delete_one({"_id": user["_id"]})
if duplicate_count == 0:
print("\n✓ No duplicate users found")
else:
print(f"\n✓ Removed {duplicate_count} duplicate users")
# ========== STEP 2: MIGRATE ENTRIES ==========
print("\n" + "=" * 70)
print("STEP 2: Migrating Entries (userId string → ObjectId, add entryDate)")
print("=" * 70)
total_entries = db.entries.count_documents({})
entries_updated = 0
entries_with_issues = []
print(f"\nTotal entries to process: {total_entries}\n")
for entry in db.entries.find():
try:
entry_id = entry["_id"]
old_user_id_str = entry.get("userId", "")
# Convert userId: string → ObjectId
if isinstance(old_user_id_str, str):
# Check if this userId is in the duplicate mapping
if old_user_id_str in user_mapping:
new_user_id = user_mapping[old_user_id_str]
print(f" → Entry {entry_id}: userId mapped {old_user_id_str[:8]}... → {str(new_user_id)[:8]}...")
print(
f" → Entry {entry_id}: userId mapped {old_user_id_str[:8]}... → {str(new_user_id)[:8]}...")
else:
new_user_id = ObjectId(old_user_id_str)
update_data = {
"userId": new_user_id,
}
@@ -110,14 +111,15 @@ def migrate_data():
# Already an ObjectId
new_user_id = old_user_id_str
update_data = {}
# Add entryDate if missing (default to createdAt)
if "entryDate" not in entry:
entry_date = entry.get("createdAt", datetime.utcnow())
# Set to start of day
entry_date = entry_date.replace(hour=0, minute=0, second=0, microsecond=0)
entry_date = entry_date.replace(
hour=0, minute=0, second=0, microsecond=0)
update_data["entryDate"] = entry_date
# Add encryption metadata if missing
if "encryption" not in entry:
update_data["encryption"] = {
@@ -125,7 +127,7 @@ def migrate_data():
"iv": None,
"algorithm": None
}
# Perform update if there are changes
if update_data:
update_data["updatedAt"] = datetime.utcnow()
@@ -134,61 +136,65 @@ def migrate_data():
{"$set": update_data}
)
entries_updated += 1
if entries_updated % 100 == 0:
print(f" ✓ Processed {entries_updated}/{total_entries} entries")
print(
f" ✓ Processed {entries_updated}/{total_entries} entries")
except Exception as e:
entries_with_issues.append({
"entry_id": str(entry_id),
"error": str(e)
})
print(f" ⚠ Error processing entry {entry_id}: {e}")
print(f"\n✓ Updated {entries_updated}/{total_entries} entries")
if entries_with_issues:
print(f"\n⚠ {len(entries_with_issues)} entries had issues:")
for issue in entries_with_issues[:5]: # Show first 5
print(f" - {issue['entry_id']}: {issue['error']}")
# ========== STEP 3: VERIFY DATA INTEGRITY ==========
print("\n" + "=" * 70)
print("STEP 3: Verifying Data Integrity")
print("=" * 70)
# Check for orphaned entries (userId doesn't exist in users)
orphaned_count = 0
users_ids = set(str(u["_id"]) for u in db.users.find({}, {"_id": 1}))
for entry in db.entries.find({}, {"userId": 1}):
user_id = entry.get("userId")
if isinstance(user_id, ObjectId):
user_id = str(user_id)
if user_id not in users_ids:
orphaned_count += 1
print(f"\nUsers collection: {db.users.count_documents({})}")
print(f"Entries collection: {db.entries.count_documents({})}")
if orphaned_count > 0:
print(f"\n⚠ WARNING: Found {orphaned_count} orphaned entries (no corresponding user)")
print(
f"\n⚠ WARNING: Found {orphaned_count} orphaned entries (no corresponding user)")
else:
print(f"✓ All entries have valid user references")
# Sample entry check
sample_entry = db.entries.find_one()
if sample_entry:
print(f"\nSample entry structure:")
print(f" _id (entry): {sample_entry['_id']} (ObjectId: {isinstance(sample_entry['_id'], ObjectId)})")
print(f" userId: {sample_entry.get('userId')} (ObjectId: {isinstance(sample_entry.get('userId'), ObjectId)})")
print(
f" _id (entry): {sample_entry['_id']} (ObjectId: {isinstance(sample_entry['_id'], ObjectId)})")
print(
f" userId: {sample_entry.get('userId')} (ObjectId: {isinstance(sample_entry.get('userId'), ObjectId)})")
print(f" entryDate present: {'entryDate' in sample_entry}")
print(f" encryption present: {'encryption' in sample_entry}")
if "entryDate" in sample_entry:
print(f" → entryDate: {sample_entry['entryDate'].isoformat()}")
if "encryption" in sample_entry:
print(f" → encryption: {sample_entry['encryption']}")
# ========== SUMMARY ==========
print(f"\n{'='*70}")
print("✓ Migration Complete")
@@ -196,12 +202,12 @@ def migrate_data():
print(f"Duplicate users removed: {duplicate_count}")
print(f"Entries migrated: {entries_updated}")
print(f"Orphaned entries found: {orphaned_count}")
if orphaned_count == 0:
print("\n✓ Data integrity verified successfully!")
else:
print(f"\n⚠ Please review {orphaned_count} orphaned entries")
client.close()
print("\n✓ Disconnected from MongoDB")
@@ -234,12 +240,13 @@ This script modifies your MongoDB database. Before running:
if __name__ == "__main__":
rollback_warning()
response = input("\nDo you want to proceed with migration? (yes/no): ").strip().lower()
response = input(
"\nDo you want to proceed with migration? (yes/no): ").strip().lower()
if response != "yes":
print("Migration cancelled.")
sys.exit(0)
try:
migrate_data()
except Exception as e: