Fix rate limiting behind proxy, make configurable

.goreleaser.yml

@@ -4,11 +4,9 @@ before:
 builds:
   - binary: ntfy
     env:
-      - CGO_ENABLED=0
+      - CGO_ENABLED=1 # required for go-sqlite3
     goos:
       - linux
-      - windows
-      - darwin
     goarch:
       - amd64
 nfpms:

Makefile

@@ -94,6 +94,7 @@ build-snapshot:
 
 build-simple: clean
 	mkdir -p dist/ntfy_linux_amd64
+	export CGO_ENABLED=1
 	$(GO) build \
 		-o dist/ntfy_linux_amd64/ntfy \
 		-ldflags \

README.md

@@ -1,47 +1,112 @@
-# ntfy
+![ntfy](server/static/img/ntfy.png)
 
-ntfy (pronounce: *notify*) is a super simple pub-sub notification service. It allows you to send desktop and (soon) phone notifications
-via scripts. I run a free version of it on *[ntfy.sh](https://ntfy.sh)*. **No signups or cost.**
+# ntfy - simple HTTP-based pub-sub
+
+**ntfy** (pronounce: *notify*) is a simple HTTP-based [pub-sub](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern) notification service.
+It allows you to **send notifications to your phone or desktop via scripts** from any computer, entirely **without signup or cost**.
+It's also open source (as you can plainly see) if you want to run your own.
+
+I run a free version of it at **[ntfy.sh](https://ntfy.sh)**, and there's an [Android app](https://play.google.com/store/apps/details?id=io.heckel.ntfy)
+too.
 
 ## Usage
 
-### Subscribe to a topic
-Topics are created on the fly by subscribing to them. You can create and subscribe to a topic either in a web UI, or in 
-your own app by subscribing to an [SSE](https://en.wikipedia.org/wiki/Server-sent_events)/[EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource),
-or a JSON or raw feed.  
+### Publishing messages
 
-Because there is no sign-up, **the topic is essentially a password**, so pick something that's not easily guessable.  
+Publishing messages can be done via PUT or POST using. Topics are created on the fly by subscribing or publishing to them.
+Because there is no sign-up, **the topic is essentially a password**, so pick something that's not easily guessable.
 
-Here's how you can create a topic `mytopic`, subscribe to it topic and wait for events. This is using `curl`, but you
-can use any library that can do HTTP GETs:
+Here's an example showing how to publish a message using `curl`:
 
-```
-# Subscribe to "mytopic" and output one message per line (\n are replaced with a space)
-curl -s ntfy.sh/mytopic/raw
-
-# Subscribe to "mytopic" and output one JSON message per line
-curl -s ntfy.sh/mytopic/json
-
-# Subscribe to "mytopic" and output an SSE stream (supported via JS/EventSource)
-curl -s ntfy.sh/mytopic/sse
-```
-
-You can easily script it to execute any command when a message arrives. This sends desktop notifications (just like 
-the web UI, but without it):
-```
-while read msg; do
-  [ -n "$msg" ] && notify-send "$msg"
-done < <(stdbuf -i0 -o0 curl -s ntfy.sh/mytopic/raw)
-```
-
-### Publish messages
-Publishing messages can be done via PUT or POST using. Here's an example using `curl`:
 ```
 curl -d "long process is done" ntfy.sh/mytopic
 ```
 
-Messages published to a non-existing topic or a topic without subscribers will not be delivered later. There is (currently)
-no buffering of any kind. If you're not listening, the message won't be delivered.
+Here's an example in JS with `fetch()` (see [full example](examples)):
+
+```
+fetch('https://ntfy.sh/mytopic', {
+  method: 'POST', // PUT works too
+  body: 'Hello from the other side.'
+})
+```
+
+### Subscribe to a topic
+You can create and subscribe to a topic either in this web UI, or in your own app by subscribing to an
+[EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource), a JSON feed, or raw feed.
+
+#### Subscribe via web
+If you subscribe to a topic via this web UI in the field below, messages published to any subscribed topic
+will show up as **desktop notification**.
+
+You can try this easily on **[ntfy.sh](https://ntfy.sh)**.
+
+#### Subscribe via phone
+You can use the [Ntfy Android App](https://play.google.com/store/apps/details?id=io.heckel.ntfy) to receive 
+notifications directly on your phone. Just like the server, this app is also [open source](https://github.com/binwiederhier/ntfy-android).
+
+#### Subscribe via your app, or via the CLI
+Using [EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) in JS, you can consume
+notifications like this (see [full example](examples)):
+
+```javascript
+const eventSource = new EventSource('https://ntfy.sh/mytopic/sse');<br/>
+eventSource.onmessage = (e) => {<br/>
+  // Do something with e.data<br/>
+};
+```
+
+You can also use the same `/sse` endpoint via `curl` or any other HTTP library:
+
+```
+$ curl -s ntfy.sh/mytopic/sse
+event: open
+data: {"id":"weSj9RtNkj","time":1635528898,"event":"open","topic":"mytopic"}
+
+data: {"id":"p0M5y6gcCY","time":1635528909,"event":"message","topic":"mytopic","message":"Hi!"}
+
+event: keepalive
+data: {"id":"VNxNIg5fpt","time":1635528928,"event":"keepalive","topic":"test"}
+```
+
+To consume JSON instead, use the `/json` endpoint, which prints one message per line:
+
+```
+$ curl -s ntfy.sh/mytopic/json
+{"id":"SLiKI64DOt","time":1635528757,"event":"open","topic":"mytopic"}
+{"id":"hwQ2YpKdmg","time":1635528741,"event":"message","topic":"mytopic","message":"Hi!"}
+{"id":"DGUDShMCsc","time":1635528787,"event":"keepalive","topic":"mytopic"}
+```
+
+Or use the `/raw` endpoint if you need something super simple (empty lines are keepalive messages):
+
+```
+$ curl -s ntfy.sh/mytopic/raw
+
+This is a notification
+```
+
+#### Message buffering and polling
+Messages are buffered in memory for a few hours to account for network interruptions of subscribers.
+You can read back what you missed by using the `since=...` query parameter. It takes either a
+duration (e.g. `10m` or `30s`) or a Unix timestamp (e.g. `1635528757`):
+
+```
+$ curl -s "ntfy.sh/mytopic/json?since=10m"
+# Same output as above, but includes messages from up to 10 minutes ago
+```
+
+You can also just poll for messages if you don't like the long-standing connection using the `poll=1`
+query parameter. The connection will end after all available messages have been read. This parameter has to be
+combined with `since=`.
+
+```
+$ curl -s "ntfy.sh/mytopic/json?poll=1&since=10m"
+# Returns messages from up to 10 minutes ago and ends the connection
+```
+
+## Examples
+There are a few usage examples in the [examples](examples) directory. I'm sure there are tons of other ways to use it.
 
 ## Installation
 Please check out the [releases page](https://github.com/binwiederhier/ntfy/releases) for binaries and
@@ -64,13 +129,13 @@ sudo apt install ntfy
 **Debian/Ubuntu** (*manual install*)**:**
 ```bash
 sudo apt install tmux
-wget https://github.com/binwiederhier/ntfy/releases/download/v1.1.2/ntfy_1.1.2_amd64.deb
-dpkg -i ntfy_1.1.2_amd64.deb
+wget https://github.com/binwiederhier/ntfy/releases/download/v1.2.0/ntfy_1.2.0_amd64.deb
+dpkg -i ntfy_1.2.0_amd64.deb
 ```
 
 **Fedora/RHEL/CentOS:**
 ```bash
-rpm -ivh https://github.com/binwiederhier/ntfy/releases/download/v1.1.2/ntfy_1.1.2_amd64.rpm
+rpm -ivh https://github.com/binwiederhier/ntfy/releases/download/v1.2.0/ntfy_1.2.0_amd64.rpm
 ```
 
 **Docker:**
@@ -85,8 +150,8 @@ go get -u heckel.io/ntfy
 
 **Manual install** (*any x86_64-based Linux*)**:**
 ```bash
-wget https://github.com/binwiederhier/ntfy/releases/download/v1.1.2/ntfy_1.1.2_linux_x86_64.tar.gz
-sudo tar -C /usr/bin -zxf ntfy_1.1.2_linux_x86_64.tar.gz ntfy
+wget https://github.com/binwiederhier/ntfy/releases/download/v1.2.0/ntfy_1.2.0_linux_x86_64.tar.gz
+sudo tar -C /usr/bin -zxf ntfy_1.2.0_linux_x86_64.tar.gz ntfy
 ./ntfy
 ```
 
@@ -104,7 +169,6 @@ To build releases, I use [GoReleaser](https://goreleaser.com/). If you have that
 ## TODO
 - add HTTPS
 - make limits configurable
-- limit max number of subscriptions 
 
 ## Contributing
 I welcome any and all contributions. Just create a PR or an issue.
@@ -116,4 +180,6 @@ Third party libraries and resources:
 * [github.com/urfave/cli/v2](https://github.com/urfave/cli/v2) (MIT) is used to drive the CLI
 * [Mixkit sound](https://mixkit.co/free-sound-effects/notification/) (Mixkit Free License) used as notification sound
 * [Lato Font](https://www.latofonts.com/) (OFL) is used as a font in the Web UI
-* [GoReleaser](https://goreleaser.com/) (MIT) is used to create releases 
+* [GoReleaser](https://goreleaser.com/) (MIT) is used to create releases
+* [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) (MIT) is used to provide the persistent message cache
+* [Firebase Admin SDK](https://github.com/firebase/firebase-admin-go) (Apache 2.0) is used to send FCM messages

cmd/app.go

@@ -19,11 +19,16 @@ func New() *cli.App {
 	flags := []cli.Flag{
 		&cli.StringFlag{Name: "config", Aliases: []string{"c"}, EnvVars: []string{"NTFY_CONFIG_FILE"}, Value: "/etc/ntfy/config.yml", DefaultText: "/etc/ntfy/config.yml", Usage: "config file"},
 		altsrc.NewStringFlag(&cli.StringFlag{Name: "listen-http", Aliases: []string{"l"}, EnvVars: []string{"NTFY_LISTEN_HTTP"}, Value: config.DefaultListenHTTP, Usage: "ip:port used to as listen address"}),
-		altsrc.NewStringFlag(&cli.StringFlag{Name: "cache-file", Aliases: []string{"C"}, EnvVars: []string{"NTFY_CACHE_FILE"}, Usage: "cache file used for message caching"}),
 		altsrc.NewStringFlag(&cli.StringFlag{Name: "firebase-key-file", Aliases: []string{"F"}, EnvVars: []string{"NTFY_FIREBASE_KEY_FILE"}, Usage: "Firebase credentials file; if set additionally publish to FCM topic"}),
-		altsrc.NewDurationFlag(&cli.DurationFlag{Name: "message-buffer-duration", Aliases: []string{"b"}, EnvVars: []string{"NTFY_MESSAGE_BUFFER_DURATION"}, Value: config.DefaultMessageBufferDuration, Usage: "buffer messages in memory for this time to allow `since` requests"}),
-		altsrc.NewDurationFlag(&cli.DurationFlag{Name: "keepalive-interval", Aliases: []string{"k"}, EnvVars: []string{"NTFY_KEEPALIVE_INTERVAL"}, Value: config.DefaultKeepaliveInterval, Usage: "default interval of keepalive messages"}),
-		altsrc.NewDurationFlag(&cli.DurationFlag{Name: "manager-interval", Aliases: []string{"m"}, EnvVars: []string{"NTFY_MANAGER_INTERVAL"}, Value: config.DefaultManagerInterval, Usage: "default interval of for message pruning and stats printing"}),
+		altsrc.NewStringFlag(&cli.StringFlag{Name: "cache-file", Aliases: []string{"C"}, EnvVars: []string{"NTFY_CACHE_FILE"}, Usage: "cache file used for message caching"}),
+		altsrc.NewDurationFlag(&cli.DurationFlag{Name: "cache-duration", Aliases: []string{"b"}, EnvVars: []string{"NTFY_CACHE_DURATION"}, Value: config.DefaultCacheDuration, Usage: "buffer messages for this time to allow `since` requests"}),
+		altsrc.NewDurationFlag(&cli.DurationFlag{Name: "keepalive-interval", Aliases: []string{"k"}, EnvVars: []string{"NTFY_KEEPALIVE_INTERVAL"}, Value: config.DefaultKeepaliveInterval, Usage: "interval of keepalive messages"}),
+		altsrc.NewDurationFlag(&cli.DurationFlag{Name: "manager-interval", Aliases: []string{"m"}, EnvVars: []string{"NTFY_MANAGER_INTERVAL"}, Value: config.DefaultManagerInterval, Usage: "interval of for message pruning and stats printing"}),
+		altsrc.NewIntFlag(&cli.IntFlag{Name: "global-topic-limit", Aliases: []string{"T"}, EnvVars: []string{"NTFY_GLOBAL_TOPIC_LIMIT"}, Value: config.DefaultGlobalTopicLimit, Usage: "total number of topics allowed"}),
+		altsrc.NewIntFlag(&cli.IntFlag{Name: "visitor-subscription-limit", Aliases: []string{"V"}, EnvVars: []string{"NTFY_VISITOR_SUBSCRIPTION_LIMIT"}, Value: config.DefaultVisitorSubscriptionLimit, Usage: "number of subscriptions per visitor"}),
+		altsrc.NewIntFlag(&cli.IntFlag{Name: "visitor-request-limit-burst", Aliases: []string{"B"}, EnvVars: []string{"NTFY_VISITOR_REQUEST_LIMIT_BURST"}, Value: config.DefaultVisitorRequestLimitBurst, Usage: "initial limit of requests per visitor"}),
+		altsrc.NewDurationFlag(&cli.DurationFlag{Name: "visitor-request-limit-replenish", Aliases: []string{"R"}, EnvVars: []string{"NTFY_VISITOR_REQUEST_LIMIT_REPLENISH"}, Value: config.DefaultVisitorRequestLimitReplenish, Usage: "interval at which burst limit is replenished (one per x)"}),
+		altsrc.NewBoolFlag(&cli.BoolFlag{Name: "behind-proxy", Aliases: []string{"P"}, EnvVars: []string{"NTFY_BEHIND_PROXY"}, Value: false, Usage: "if set, use X-Forwarded-For header to determine visitor IP address (for rate limiting)"}),
 	}
 	return &cli.App{
 		Name:                   "ntfy",
@@ -45,11 +50,16 @@ func New() *cli.App {
 func execRun(c *cli.Context) error {
 	// Read all the options
 	listenHTTP := c.String("listen-http")
-	cacheFile := c.String("cache-file")
 	firebaseKeyFile := c.String("firebase-key-file")
-	messageBufferDuration := c.Duration("message-buffer-duration")
+	cacheFile := c.String("cache-file")
+	cacheDuration := c.Duration("cache-duration")
 	keepaliveInterval := c.Duration("keepalive-interval")
 	managerInterval := c.Duration("manager-interval")
+	globalTopicLimit := c.Int("global-topic-limit")
+	visitorSubscriptionLimit := c.Int("visitor-subscription-limit")
+	visitorRequestLimitBurst := c.Int("visitor-request-limit-burst")
+	visitorRequestLimitReplenish := c.Duration("visitor-request-limit-replenish")
+	behindProxy := c.Bool("behind-proxy")
 
 	// Check values
 	if firebaseKeyFile != "" && !util.FileExists(firebaseKeyFile) {
@@ -58,17 +68,22 @@ func execRun(c *cli.Context) error {
 		return errors.New("keepalive interval cannot be lower than five seconds")
 	} else if managerInterval < 5*time.Second {
 		return errors.New("manager interval cannot be lower than five seconds")
-	} else if messageBufferDuration < managerInterval {
-		return errors.New("message buffer duration cannot be lower than manager interval")
+	} else if cacheDuration < managerInterval {
+		return errors.New("cache duration cannot be lower than manager interval")
 	}
 
 	// Run server
 	conf := config.New(listenHTTP)
-	conf.CacheFile = cacheFile
 	conf.FirebaseKeyFile = firebaseKeyFile
-	conf.MessageBufferDuration = messageBufferDuration
+	conf.CacheFile = cacheFile
+	conf.CacheDuration = cacheDuration
 	conf.KeepaliveInterval = keepaliveInterval
 	conf.ManagerInterval = managerInterval
+	conf.GlobalTopicLimit = globalTopicLimit
+	conf.VisitorSubscriptionLimit = visitorSubscriptionLimit
+	conf.VisitorRequestLimitBurst = visitorRequestLimitBurst
+	conf.VisitorRequestLimitReplenish = visitorRequestLimitReplenish
+	conf.BehindProxy = behindProxy
 	s, err := server.New(conf)
 	if err != nil {
 		log.Fatalln(err)

config/config.go

@@ -2,55 +2,56 @@
 package config
 
 import (
-	"golang.org/x/time/rate"
 	"time"
 )
 
 // Defines default config settings
 const (
-	DefaultListenHTTP            = ":80"
-	DefaultMessageBufferDuration = 12 * time.Hour
-	DefaultKeepaliveInterval     = 30 * time.Second
-	DefaultManagerInterval       = time.Minute
+	DefaultListenHTTP        = ":80"
+	DefaultCacheDuration     = 12 * time.Hour
+	DefaultKeepaliveInterval = 30 * time.Second
+	DefaultManagerInterval   = time.Minute
 )
 
 // Defines all the limits
-// - request limit: max number of PUT/GET/.. requests (here: 50 requests bucket, replenished at a rate of 1 per second)
 // - global topic limit: max number of topics overall
-// - subscription limit: max number of subscriptions (active HTTP connections) per per-visitor/IP
-var (
-	defaultGlobalTopicLimit         = 5000
-	defaultVisitorRequestLimit      = rate.Every(time.Second)
-	defaultVisitorRequestLimitBurst = 50
-	defaultVisitorSubscriptionLimit = 30
+// - per visistor request limit: max number of PUT/GET/.. requests (here: 60 requests bucket, replenished at a rate of one per 10 seconds)
+// - per visistor subscription limit: max number of subscriptions (active HTTP connections) per per-visitor/IP
+const (
+	DefaultGlobalTopicLimit             = 5000
+	DefaultVisitorRequestLimitBurst     = 60
+	DefaultVisitorRequestLimitReplenish = 10 * time.Second
+	DefaultVisitorSubscriptionLimit     = 30
 )
 
 // Config is the main config struct for the application. Use New to instantiate a default config struct.
 type Config struct {
-	ListenHTTP               string
-	CacheFile          string
-	FirebaseKeyFile          string
-	MessageBufferDuration    time.Duration
-	KeepaliveInterval        time.Duration
-	ManagerInterval          time.Duration
-	GlobalTopicLimit         int
-	VisitorRequestLimit      rate.Limit
-	VisitorRequestLimitBurst int
-	VisitorSubscriptionLimit int
+	ListenHTTP                   string
+	FirebaseKeyFile              string
+	CacheFile                    string
+	CacheDuration                time.Duration
+	KeepaliveInterval            time.Duration
+	ManagerInterval              time.Duration
+	GlobalTopicLimit             int
+	VisitorRequestLimitBurst     int
+	VisitorRequestLimitReplenish time.Duration
+	VisitorSubscriptionLimit     int
+	BehindProxy                  bool
 }
 
 // New instantiates a default new config
 func New(listenHTTP string) *Config {
 	return &Config{
-		ListenHTTP:               listenHTTP,
-		CacheFile: "",
-		FirebaseKeyFile:          "",
-		MessageBufferDuration:    DefaultMessageBufferDuration,
-		KeepaliveInterval:        DefaultKeepaliveInterval,
-		ManagerInterval:          DefaultManagerInterval,
-		GlobalTopicLimit:         defaultGlobalTopicLimit,
-		VisitorRequestLimit:      defaultVisitorRequestLimit,
-		VisitorRequestLimitBurst: defaultVisitorRequestLimitBurst,
-		VisitorSubscriptionLimit: defaultVisitorSubscriptionLimit,
+		ListenHTTP:                   listenHTTP,
+		FirebaseKeyFile:              "",
+		CacheFile:                    "",
+		CacheDuration:                DefaultCacheDuration,
+		KeepaliveInterval:            DefaultKeepaliveInterval,
+		ManagerInterval:              DefaultManagerInterval,
+		GlobalTopicLimit:             DefaultGlobalTopicLimit,
+		VisitorRequestLimitBurst:     DefaultVisitorRequestLimitBurst,
+		VisitorRequestLimitReplenish: DefaultVisitorRequestLimitReplenish,
+		VisitorSubscriptionLimit:     DefaultVisitorSubscriptionLimit,
+		BehindProxy:                  false,
 	}
 }

config/config.yml

@@ -10,16 +10,45 @@
 #
 # firebase-key-file: <filename>
 
+# If set, messages are cached in a local SQLite database instead of only in-memory. This
+# allows for service restarts without losing messages in support of the since= parameter.
+#
+# cache-file: <filename>
+
 # Duration for which messages will be buffered before they are deleted.
 # This is required to support the "since=..." and "poll=1" parameter.
 #
-# message-buffer-duration: 12h
+# cache-duration: 12h
 
 # Interval in which keepalive messages are sent to the client. This is to prevent
 # intermediaries closing the connection for inactivity.
 #
 # keepalive-interval: 30s
 
-# Interval in which the manager prunes old messages, deletes topics and prints the stats.
+# Interval in which the manager prunes old messages, deletes topics
+# and prints the stats.
 #
 # manager-interval: 1m
+
+# Rate limiting: Total number of topics before the server rejects new topics.
+#
+# global-topic-limit: 5000
+
+# Rate limiting: Number of subscriptions per visitor (IP address)
+#
+# visitor-subscription-limit: 30
+
+# Rate limiting: Allowed GET/PUT/POST requests per second, per visitor:
+# - visitor-request-limit-burst is the initial bucket of requests each visitor has
+# - visitor-request-limit-replenish is the rate at which the bucket is refilled
+#
+# visitor-request-limit-burst: 60
+# visitor-request-limit-replenish: 10s
+
+# If set, the X-Forwarded-For header is used to determine the visitor IP address
+# instead of the remote address of the connection.
+#
+# WARNING: If you are behind a proxy, you must set this, otherwise all visitors are rate limited
+#          as if they are one.
+#
+# behind-proxy: false

server/cache_mem.go

@@ -7,8 +7,8 @@ import (
 )
 
 type memCache struct {
-	messages    map[string][]*message
-	mu sync.Mutex
+	messages map[string][]*message
+	mu       sync.Mutex
 }
 
 var _ cache = (*memCache)(nil)

server/cache_sqlite.go

@@ -19,8 +19,8 @@ const (
 		CREATE INDEX IF NOT EXISTS idx_topic ON messages (topic);
 		COMMIT;
 	`
-	insertMessageQuery = `INSERT INTO messages (id, time, topic, message) VALUES (?, ?, ?, ?)`
-	pruneMessagesQuery = `DELETE FROM messages WHERE time < ?`
+	insertMessageQuery           = `INSERT INTO messages (id, time, topic, message) VALUES (?, ?, ?, ?)`
+	pruneMessagesQuery           = `DELETE FROM messages WHERE time < ?`
 	selectMessagesSinceTimeQuery = `
 		SELECT id, time, message 
 		FROM messages 
@@ -46,7 +46,7 @@ func newSqliteCache(filename string) (*sqliteCache, error) {
 		return nil, err
 	}
 	return &sqliteCache{
-		db:     db,
+		db: db,
 	}, nil
 }
 
@@ -122,6 +122,6 @@ func (s *sqliteCache) Topics() (map[string]*topic, error) {
 }
 
 func (c *sqliteCache) Prune(keep time.Duration) error {
-	_, err := c.db.Exec(pruneMessagesQuery, time.Now().Add(-1 * keep).Unix())
+	_, err := c.db.Exec(pruneMessagesQuery, time.Now().Add(-1*keep).Unix())
 	return err
 }

server/index.html

@@ -33,10 +33,14 @@
     <h1><img src="static/img/ntfy.png" alt="ntfy"/><br/>ntfy.sh - simple HTTP-based pub-sub</h1>
     <p>
         <b>ntfy</b> (pronounce: <i>notify</i>) is a simple HTTP-based <a href="https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern">pub-sub</a> notification service.
-        It allows you to send <b>desktop notifications via scripts from any computer</b>, entirely <b>without signup or cost</b>.
-        It's also <a href="https://github.com/binwiederhier/ntfy">open source</a> if you want to run your own.
+        It allows you to send notifications <a href="https://play.google.com/store/apps/details?id=io.heckel.ntfy">to your phone</a> or desktop via scripts from any computer,
+        entirely <b>without signup or cost</b>. It's also <a href="https://github.com/binwiederhier/ntfy">open source</a> if you want to run your own.
+    </p>
+    <p>
+        There are many ways to use ntfy. You can send yourself messages for all sorts of things: When a long process finishes or fails (a backup, a long rsync job, ...),
+        or to notify yourself when somebody logs into your server(s). Or you may want to use it in your own app to distribute messages to subscribed clients.
+        Endless possibilities 😀.
     </p>
-    <p id="error"></p>
 
     <h2>Publishing messages</h2>
     <p>
@@ -65,26 +69,29 @@
         <a href="https://developer.mozilla.org/en-US/docs/Web/API/EventSource">EventSource</a>, a JSON feed, or raw feed.
     </p>
 
-    <h3>Subscribe via web</h3>
-    <p>
-        If you subscribe to a topic via this web UI in the field below, messages published to any subscribed topic
-        will show up as <b>desktop notification</b>.
-    </p>
-    <form id="subscribeForm">
+    <div id="subscribeBox">
+        <h3>Subscribe in this Web UI</h3>
+        <p id="error"></p>
         <p>
-            <label for="topicField">Subscribe to topic:</label>
-            <input type="text" id="topicField" placeholder="Letters, numbers, _ and -"  pattern="[-_A-Za-z]{1,64}" />
-            <input type="submit" id="subscribeButton" value="Subscribe" />
+            Subscribe to topics here and receive messages as <b>desktop notification</b>. Topics are not password-protected,
+            so choose a name that's not easy to guess. Once subscribed, you can publish messages via PUT/POST.
         </p>
-    </form>
-    <p id="topicsHeader">Topics:</p>
-    <ul id="topicsList"></ul>
-    <audio id="notifySound" src="static/sound/mixkit-message-pop-alert-2354.mp3"></audio>
+        <form id="subscribeForm">
+            <p>
+                <b>Topic:</b><br/>
+                <input type="text" id="topicField" autocomplete="off" placeholder="Topic name, e.g. phil_alerts"  pattern="[-_A-Za-z]{1,64}" />
+                <button id="subscribeButton">Subscribe</button>
+            </p>
+            <p id="topicsHeader"><b>Subscribed topics:</b></p>
+            <ul id="topicsList"></ul>
+        </form>
+        <audio id="notifySound" src="static/sound/mixkit-message-pop-alert-2354.mp3"></audio>
+    </div>
 
-    <h3>Subscribe via phone</h3>
+    <h3>Subscribe via Android App</h3>
     <p>
-        Once it's approved, you can use the <b>Ntfy Android App</b> to receive notifications directly on your phone. Just like
-        the server, this app is also <a href="https://github.com/binwiederhier/ntfy-android">open source</a>.
+        You can use the <a href="https://play.google.com/store/apps/details?id=io.heckel.ntfy">Ntfy Android App</a>
+        to receive notifications directly on your phone. Just like the server, this app is also <a href="https://github.com/binwiederhier/ntfy-android">open source</a>.
     </p>
 
     <h3>Subscribe via your app, or via the CLI</h3>
@@ -128,6 +135,14 @@
         <br/>
         This is a notification
     </code>
+    <p class="smallMarginBottom">
+        Here's an example of how to use this endpoint to send desktop notifications for every incoming message:
+    </p>
+    <code>
+        while read msg; do<br/>
+        &nbsp;&nbsp;[ -n "$msg" ] && notify-send "$msg"<br/>
+        done < <(stdbuf -i0 -o0 curl -s ntfy.sh/mytopic/raw)
+    </code>
 
     <h3>Message buffering and polling</h3>
     <p class="smallMarginBottom">
@@ -177,18 +192,16 @@
         In addition to caching messages locally and delivering them to long-polling subscribers, all messages are also
         published to Firebase Cloud Messaging (FCM) (if <tt>FirebaseKeyFile</tt> is set, which it is on ntfy.sh). This
         is to facilitate instant notifications on Android. I tried really, really hard to avoid using FCM, but newer
-        versions of Android made it impossible to implement <a href="https://developer.android.com/guide/background">background services</a>>.
-        I'm sorry.
+        versions of Android made it impossible to implement <a href="https://developer.android.com/guide/background">background services</a>.
     </p>
 
     <h2>Privacy policy</h2>
     <p>
         Neither the server nor the app record any personal information, or share any of the messages and topics with
-        any outside service. All data is exclusively used to make the service function properly. The notable exception
+        any outside service. All data is exclusively used to make the service function properly. The one exception
         is the Firebase Cloud Messaging (FCM) service, which is required to provide instant Android notifications (see
         FAQ for details).
     </p>
-
     <p>
         The web server does not log or otherwise store request paths, remote IP addresses or even topics or messages,
         aside from a short on-disk cache (up to a day) to support service restarts.

server/server.go

@@ -47,7 +47,7 @@ func (e errHTTP) Error() string {
 }
 
 const (
-	messageLimit = 1024
+	messageLimit = 512
 )
 
 var (
@@ -86,6 +86,11 @@ func New(conf *config.Config) (*Server, error) {
 	if err != nil {
 		return nil, err
 	}
+	for _, t := range topics {
+		if firebaseSubscriber != nil {
+			t.Subscribe(firebaseSubscriber)
+		}
+	}
 	return &Server{
 		config:   conf,
 		cache:    cache,
@@ -154,24 +159,22 @@ func (s *Server) handle(w http.ResponseWriter, r *http.Request) {
 }
 
 func (s *Server) handleInternal(w http.ResponseWriter, r *http.Request) error {
-	v := s.visitor(r.RemoteAddr)
-	if err := v.RequestAllowed(); err != nil {
-		return err
-	}
 	if r.Method == http.MethodGet && r.URL.Path == "/" {
 		return s.handleHome(w, r)
+	} else if r.Method == http.MethodHead && r.URL.Path == "/" {
+		return s.handleEmpty(w, r)
 	} else if r.Method == http.MethodGet && staticRegex.MatchString(r.URL.Path) {
 		return s.handleStatic(w, r)
-	} else if (r.Method == http.MethodPut || r.Method == http.MethodPost) && topicRegex.MatchString(r.URL.Path) {
-		return s.handlePublish(w, r, v)
-	} else if r.Method == http.MethodGet && jsonRegex.MatchString(r.URL.Path) {
-		return s.handleSubscribeJSON(w, r, v)
-	} else if r.Method == http.MethodGet && sseRegex.MatchString(r.URL.Path) {
-		return s.handleSubscribeSSE(w, r, v)
-	} else if r.Method == http.MethodGet && rawRegex.MatchString(r.URL.Path) {
-		return s.handleSubscribeRaw(w, r, v)
 	} else if r.Method == http.MethodOptions {
 		return s.handleOptions(w, r)
+	} else if (r.Method == http.MethodPut || r.Method == http.MethodPost) && topicRegex.MatchString(r.URL.Path) {
+		return s.withRateLimit(w, r, s.handlePublish)
+	} else if r.Method == http.MethodGet && jsonRegex.MatchString(r.URL.Path) {
+		return s.withRateLimit(w, r, s.handleSubscribeJSON)
+	} else if r.Method == http.MethodGet && sseRegex.MatchString(r.URL.Path) {
+		return s.withRateLimit(w, r, s.handleSubscribeSSE)
+	} else if r.Method == http.MethodGet && rawRegex.MatchString(r.URL.Path) {
+		return s.withRateLimit(w, r, s.handleSubscribeRaw)
 	}
 	return errHTTPNotFound
 }
@@ -181,6 +184,10 @@ func (s *Server) handleHome(w http.ResponseWriter, r *http.Request) error {
 	return err
 }
 
+func (s *Server) handleEmpty(w http.ResponseWriter, r *http.Request) error {
+	return nil
+}
+
 func (s *Server) handleStatic(w http.ResponseWriter, r *http.Request) error {
 	http.FileServer(http.FS(webStaticFs)).ServeHTTP(w, r)
 	return nil
@@ -204,6 +211,9 @@ func (s *Server) handlePublish(w http.ResponseWriter, r *http.Request, v *visito
 		return err
 	}
 	w.Header().Set("Access-Control-Allow-Origin", "*") // CORS, allow cross-origin requests
+	if err := json.NewEncoder(w).Encode(m); err != nil {
+		return err
+	}
 	s.mu.Lock()
 	s.messages++
 	s.mu.Unlock()
@@ -360,7 +370,7 @@ func (s *Server) updateStatsAndExpire() {
 	}
 
 	// Prune cache
-	if err := s.cache.Prune(s.config.MessageBufferDuration); err != nil {
+	if err := s.cache.Prune(s.config.CacheDuration); err != nil {
 		log.Printf("error pruning cache: %s", err.Error())
 	}
 
@@ -386,15 +396,27 @@ func (s *Server) updateStatsAndExpire() {
 		s.messages, len(s.topics), subscribers, messages, len(s.visitors))
 }
 
+func (s *Server) withRateLimit(w http.ResponseWriter, r *http.Request, handler func(w http.ResponseWriter, r *http.Request, v *visitor) error) error {
+	v := s.visitor(r)
+	if err := v.RequestAllowed(); err != nil {
+		return err
+	}
+	return handler(w, r, v)
+}
+
 // visitor creates or retrieves a rate.Limiter for the given visitor.
 // This function was taken from https://www.alexedwards.net/blog/how-to-rate-limit-http-requests (MIT).
-func (s *Server) visitor(remoteAddr string) *visitor {
+func (s *Server) visitor(r *http.Request) *visitor {
 	s.mu.Lock()
 	defer s.mu.Unlock()
+	remoteAddr := r.RemoteAddr
 	ip, _, err := net.SplitHostPort(remoteAddr)
 	if err != nil {
 		ip = remoteAddr // This should not happen in real life; only in tests.
 	}
+	if s.config.BehindProxy && r.Header.Get("X-Forwarded-For") != "" {
+		ip = r.Header.Get("X-Forwarded-For")
+	}
 	v, exists := s.visitors[ip]
 	if !exists {
 		s.visitors[ip] = newVisitor(s.config)

server/static/css/app.css

@@ -58,6 +58,7 @@ code {
     border-radius: 3px;
     margin-top: 10px;
     margin-bottom: 20px;
+    overflow-x: auto;
 }
 
 /* Lato font (OFL), https://fonts.google.com/specimen/Lato#about,
@@ -87,3 +88,159 @@ code {
 #ironicCenterTagDontFreakOut {
     color: #666;
 }
+
+/* Subscribe box */
+
+button {
+    background: #3a9784;
+    border: none;
+    border-radius: 3px;
+    padding: 3px 5px;
+    color: white;
+    cursor: pointer;
+}
+
+button:hover {
+    background: #317f6f;
+    padding: 5px;
+}
+
+ul {
+    padding-left: 1em;
+    list-style-type: none;
+    padding-bottom: 0;
+    margin: 0;
+}
+
+li {
+    padding: 4px 0;
+    margin: 4px 0;
+    font-size: 0.9em;
+}
+
+/* Subscribe box SMALL SCREEN */
+@media only screen and (max-width: 1599px) {
+    #subscribeBox #subscribeForm {
+        border-left: 4px solid #3a9784;
+        padding: 10px;
+    }
+
+    #subscribeBox #topicsHeader {
+        margin-bottom: 0;
+    }
+
+    #subscribeBox input {
+        height: 24px;
+        min-width: 200px;
+        max-width: 300px;
+        border-radius: 3px;
+        border: none;
+        border-bottom: 1px solid #aaa;
+        font-size: 0.8em;
+    }
+
+    #subscribeBox input:focus {
+        border-bottom: 2px solid #3a9784;
+        outline: none;
+    }
+
+    #subscribeBox ul {
+        margin: 0;
+        padding: 0;
+    }
+
+    #subscribeBox li {
+        margin: 3px 0;
+        padding: 0;
+    }
+
+    #subscribeBox li img {
+        width: 15px;
+        height: 15px;
+        vertical-align: bottom;
+    }
+
+    #subscribeBox button {
+        font-size: 0.8em;
+        background: #3a9784;
+        border-radius: 3px;
+        padding: 5px;
+        color: white;
+        cursor: pointer;
+    }
+
+    #subscribeBox button:hover {
+        background: #317f6f;
+    }
+}
+
+/* Subscribe box BIG SCREEN */
+@media only screen and (min-width: 1600px) {
+    #subscribeBox {
+        position: fixed;
+        top: 170px;
+        right: 10px;
+        width: 300px;
+        border-left: 4px solid #3a9784;
+        padding: 10px;
+    }
+
+    #subscribeBox h3 {
+        margin-top: 0;
+        margin-bottom: 5px;
+        font-size: 1.1em;
+    }
+
+    #subscribeBox #topicsHeader {
+        margin-bottom: 0;
+    }
+
+    #subscribeBox p {
+        font-size: 0.9em;
+        margin-bottom: 10px;
+    }
+
+    #subscribeBox ul {
+        margin: 0;
+        padding: 0;
+    }
+
+    #subscribeBox input {
+        height: 18px;
+        border-radius: 3px;
+        border: none;
+        border-bottom: 1px solid #aaa;
+    }
+
+    #subscribeBox input:focus {
+        border-bottom: 2px solid #3a9784;
+        outline: none;
+    }
+
+    #subscribeBox li {
+        margin: 3px 0;
+        padding: 0;
+    }
+
+    #subscribeBox li img {
+        width: 15px;
+        height: 15px;
+        vertical-align: bottom;
+    }
+
+    #subscribeBox button {
+        font-size: 0.7em;
+        background: #3a9784;
+        border-radius: 3px;
+        padding: 5px;
+        color: white;
+        cursor: pointer;
+    }
+
+    #subscribeBox button:hover {
+        background: #317f6f;
+    }
+
+}
+
+

server/static/img/clear_black_24dp.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" height="24px" viewBox="0 0 24 24" width="24px" fill="#FFFFFF"><path d="M0 0h24v24H0V0z" fill="none"/><path d="M19 6.41L17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12 19 6.41z"/></svg>

server/static/img/send_black_24dp.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" height="24px" viewBox="0 0 24 24" width="24px" fill="#ffffff"><path d="M0 0h24v24H0z" fill="none"/><path d="M2.01 21L23 12 2.01 3 2 10l15 2-15 2z"/></svg>

server/static/js/app.js

@@ -16,7 +16,6 @@ const topicsList = document.getElementById("topicsList");
 const topicField = document.getElementById("topicField");
 const notifySound = document.getElementById("notifySound");
 const subscribeButton = document.getElementById("subscribeButton");
-const subscribeForm = document.getElementById("subscribeForm");
 const errorField = document.getElementById("error");
 
 const subscribe = (topic) => {
@@ -40,7 +39,7 @@ const subscribeInternal = (topic, delaySec) => {
         if (!topicEntry) {
             topicEntry = document.createElement('li');
             topicEntry.id = `topic-${topic}`;
-            topicEntry.innerHTML = `${topic} <button onclick="test('${topic}')">Test</button> <button onclick="unsubscribe('${topic}')">Unsubscribe</button>`;
+            topicEntry.innerHTML = `${topic} <button onclick="test('${topic}'); return false;"> <img src="static/img/send_black_24dp.svg"> Test</button> <button onclick="unsubscribe('${topic}'); return false;"> <img src="static/img/clear_black_24dp.svg"> Unsubscribe</button>`;
             topicsList.appendChild(topicEntry);
         }
         topicsHeader.style.display = '';
@@ -48,13 +47,13 @@ const subscribeInternal = (topic, delaySec) => {
         // Open event source
         let eventSource = new EventSource(`${topic}/sse`);
         eventSource.onopen = () => {
-            topicEntry.innerHTML = `${topic} <button onclick="test('${topic}')">Test</button> <button onclick="unsubscribe('${topic}')">Unsubscribe</button>`;
+            topicEntry.innerHTML = `${topic} <button onclick="test('${topic}'); return false;"> <img src="static/img/send_black_24dp.svg"> Test</button> <button onclick="unsubscribe('${topic}'); return false;"> <img src="static/img/clear_black_24dp.svg"> Unsubscribe</button>`;
             delaySec = 0; // Reset on successful connection
         };
         eventSource.onerror = (e) => {
+            topicEntry.innerHTML = `${topic} <i>(Reconnecting)</i> <button disabled="disabled">Test</button> <button onclick="unsubscribe('${topic}'); return false;">Unsubscribe</button>`;
+            eventSource.close();
             const newDelaySec = (delaySec + 5 <= 15) ? delaySec + 5 : 15;
-            topicEntry.innerHTML = `${topic} <i>(Reconnecting in ${newDelaySec}s ...)</i> <button disabled="disabled">Test</button> <button onclick="unsubscribe('${topic}')">Unsubscribe</button>`;
-            eventSource.close()
             subscribeInternal(topic, newDelaySec);
         };
         eventSource.onmessage = (e) => {
@@ -83,7 +82,7 @@ const unsubscribe = (topic) => {
 const test = (topic) => {
     fetch(`/${topic}`, {
         method: 'PUT',
-        body: `This is a test notification`
+        body: `This is a test notification sent by the ntfy.sh Web UI at ${new Date().toString()}.`
     })
 };
 
@@ -101,7 +100,7 @@ const showNotificationDeniedError = () => {
     showError("You have blocked desktop notifications for this website. Please unblock them and refresh to use the web-based desktop notifications.");
 };
 
-subscribeForm.onsubmit = function () {
+subscribeButton.onclick = function () {
     if (!topicField.value) {
         return false;
     }

server/visitor.go

@@ -24,7 +24,7 @@ type visitor struct {
 func newVisitor(conf *config.Config) *visitor {
 	return &visitor{
 		config:        conf,
-		limiter:       rate.NewLimiter(conf.VisitorRequestLimit, conf.VisitorRequestLimitBurst),
+		limiter:       rate.NewLimiter(rate.Every(conf.VisitorRequestLimitReplenish), conf.VisitorRequestLimitBurst),
 		subscriptions: util.NewLimiter(int64(conf.VisitorSubscriptionLimit)),
 		seen:          time.Now(),
 	}