1 // FIXME: if an exception is thrown, we shouldn't necessarily cache... 2 // FIXME: there's some annoying duplication of code in the various versioned mains 3 4 // add the Range header in there too. should return 206 5 6 // FIXME: cgi per-request arena allocator 7 8 // i need to add a bunch of type templates for validations... mayne @NotNull or NotNull! 9 10 // FIXME: I might make a cgi proxy class which can change things; the underlying one is still immutable 11 // but the later one can edit and simplify the api. You'd have to use the subclass tho! 12 13 /* 14 void foo(int f, @("test") string s) {} 15 16 void main() { 17 static if(is(typeof(foo) Params == __parameters)) 18 //pragma(msg, __traits(getAttributes, Params[0])); 19 pragma(msg, __traits(getAttributes, Params[1..2])); 20 else 21 pragma(msg, "fail"); 22 } 23 */ 24 25 // Note: spawn-fcgi can help with fastcgi on nginx 26 27 // FIXME: to do: add openssl optionally 28 // make sure embedded_httpd doesn't send two answers if one writes() then dies 29 30 // future direction: websocket as a separate process that you can sendfile to for an async passoff of those long-lived connections 31 32 /* 33 Session manager process: it spawns a new process, passing a 34 command line argument, to just be a little key/value store 35 of some serializable struct. On Windows, it CreateProcess. 36 On Linux, it can just fork or maybe fork/exec. The session 37 key is in a cookie. 38 39 Server-side event process: spawns an async manager. You can 40 push stuff out to channel ids and the clients listen to it. 41 42 websocket process: spawns an async handler. They can talk to 43 each other or get info from a cgi request. 44 45 Tempting to put web.d 2.0 in here. It would: 46 * map urls and form generation to functions 47 * have data presentation magic 48 * do the skeleton stuff like 1.0 49 * auto-cache generated stuff in files (at least if pure?) 50 * introspect functions in json for consumers 51 52 53 https://linux.die.net/man/3/posix_spawn 54 */ 55 56 /++ 57 Provides a uniform server-side API for CGI, FastCGI, SCGI, and HTTP web applications. 58 59 --- 60 import arsd.cgi; 61 62 // Instead of writing your own main(), you should write a function 63 // that takes a Cgi param, and use mixin GenericMain 64 // for maximum compatibility with different web servers. 65 void hello(Cgi cgi) { 66 cgi.setResponseContentType("text/plain"); 67 68 if("name" in cgi.get) 69 cgi.write("Hello, " ~ cgi.get["name"]); 70 else 71 cgi.write("Hello, world!"); 72 } 73 74 mixin GenericMain!hello; 75 --- 76 77 Test on console (works in any interface mode): 78 $(CONSOLE 79 $ ./cgi_hello GET / name=whatever 80 ) 81 82 If using http version (default on `dub` builds, or on custom builds when passing `-version=embedded_httpd` to dmd): 83 $(CONSOLE 84 $ ./cgi_hello --port 8080 85 # now you can go to http://localhost:8080/?name=whatever 86 ) 87 88 89 Compile_versions: 90 91 If you are using `dub`, use: 92 93 ```sdlang 94 subConfiguration "arsd-official:cgi" "VALUE_HERE" 95 ``` 96 97 or to dub.json: 98 99 ```json 100 "subConfigurations": {"arsd-official:cgi": "VALUE_HERE"} 101 ``` 102 103 to change versions. The possible options for `VALUE_HERE` are: 104 105 $(LIST 106 * `embedded_httpd` for the embedded httpd version (built-in web server). This is the default. 107 * `cgi` for traditional cgi binaries. 108 * `fastcgi` for FastCGI builds. 109 * `scgi` for SCGI builds. 110 * `stdio_http` for speaking raw http over stdin and stdout. See [RequestServer.serveSingleHttpConnectionOnStdio] for more information. 111 ) 112 113 With dmd, use: 114 115 $(TABLE_ROWS 116 117 * + Interfaces 118 + (mutually exclusive) 119 120 * - `-version=plain_cgi` 121 - The default building the module alone without dub - a traditional, plain CGI executable will be generated. 122 * - `-version=embedded_httpd` 123 - A HTTP server will be embedded in the generated executable. This is default when building with dub. 124 * - `-version=fastcgi` 125 - A FastCGI executable will be generated. 126 * - `-version=scgi` 127 - A SCGI (SimpleCGI) executable will be generated. 128 129 * - `-version=embedded_httpd_threads` 130 - The embedded HTTP server will use a single process with a thread pool. (use instead of plain `embedded_httpd` if you want this specific implementation) 131 * - `-version=embedded_httpd_processes` 132 - The embedded HTTP server will use a prefork style process pool. (use instead of plain `embedded_httpd` if you want this specific implementation) 133 * - `-version=embedded_httpd_processes_accept_after_fork` 134 - It will call accept() in each child process, after forking. This is currently the only option, though I am experimenting with other ideas. You probably should NOT specify this right now. 135 * - `-version=stdio_http` 136 - The embedded HTTP server will be spoken over stdin and stdout. 137 138 * + Tweaks 139 + (can be used together with others) 140 141 * - `-version=cgi_with_websocket` 142 - The CGI class has websocket server support. 143 144 * - `-version=with_openssl` 145 - not currently used 146 * - `-version=cgi_embedded_sessions` 147 - The session server will be embedded in the cgi.d server process 148 * - `-version=cgi_session_server_process` 149 - The session will be provided in a separate process, provided by cgi.d. 150 ) 151 152 Compile_and_run: 153 154 For CGI, `dmd yourfile.d cgi.d` then put the executable in your cgi-bin directory. 155 156 For FastCGI: `dmd yourfile.d cgi.d -version=fastcgi` and run it. spawn-fcgi helps on nginx. You can put the file in the directory for Apache. On IIS, run it with a port on the command line (this causes it to call FCGX_OpenSocket, which can work on nginx too). 157 158 For SCGI: `dmd yourfile.d cgi.d -version=scgi` and run the executable, providing a port number on the command line. 159 160 For an embedded HTTP server, run `dmd yourfile.d cgi.d -version=embedded_httpd` and run the generated program. It listens on port 8085 by default. You can change this on the command line with the --port option when running your program. 161 162 You can also simulate a request by passing parameters on the command line, like: 163 164 $(CONSOLE 165 ./yourprogram GET / name=adr 166 ) 167 168 And it will print the result to stdout. 169 170 CGI_Setup_tips: 171 172 On Apache, you may do `SetHandler cgi-script` in your `.htaccess` file. 173 174 Integration_tips: 175 176 cgi.d works well with dom.d for generating html. You may also use web.d for other utilities and automatic api wrapping. 177 178 dom.d usage: 179 180 --- 181 import arsd.cgi; 182 import arsd.dom; 183 184 void hello_dom(Cgi cgi) { 185 auto document = new Document(); 186 187 static import std.file; 188 // parse the file in strict mode, requiring it to be well-formed UTF-8 XHTML 189 // (You'll appreciate this if you've ever had to deal with a missing </div> 190 // or something in a php or erb template before that would randomly mess up 191 // the output in your browser. Just check it and throw an exception early!) 192 // 193 // You could also hard-code a template or load one at compile time with an 194 // import expression, but you might appreciate making it a regular file 195 // because that means it can be more easily edited by the frontend team and 196 // they can see their changes without needing to recompile the program. 197 // 198 // Note on CTFE: if you do choose to load a static file at compile time, 199 // you *can* parse it in CTFE using enum, which will cause it to throw at 200 // compile time, which is kinda cool too. Be careful in modifying that document, 201 // though, as it will be a static instance. You might want to clone on on demand, 202 // or perhaps modify it lazily as you print it out. (Try element.tree, it returns 203 // a range of elements which you could send through std.algorithm functions. But 204 // since my selector implementation doesn't work on that level yet, you'll find that 205 // harder to use. Of course, you could make a static list of matching elements and 206 // then use a simple e is e2 predicate... :) ) 207 document.parseUtf8(std.file.read("your_template.html"), true, true); 208 209 // fill in data using DOM functions, so placing it is in the hands of HTML 210 // and it will be properly encoded as text too. 211 // 212 // Plain html templates can't run server side logic, but I think that's a 213 // good thing - it keeps them simple. You may choose to extend the html, 214 // but I think it is best to try to stick to standard elements and fill them 215 // in with requested data with IDs or class names. A further benefit of 216 // this is the designer can also highlight data based on sources in the CSS. 217 // 218 // However, all of dom.d is available, so you can format your data however 219 // you like. You can do partial templates with innerHTML too, or perhaps better, 220 // injecting cloned nodes from a partial document. 221 // 222 // There's a lot of possibilities. 223 document["#name"].innerText = cgi.request("name", "default name"); 224 225 // send the document to the browser. The second argument to `cgi.write` 226 // indicates that this is all the data at once, enabling a few small 227 // optimizations. 228 cgi.write(document.toString(), true); 229 } 230 --- 231 232 Concepts: 233 Input: [Cgi.get], [Cgi.post], [Cgi.request], [Cgi.files], [Cgi.cookies], [Cgi.pathInfo], [Cgi.requestMethod], 234 and HTTP headers ([Cgi.headers], [Cgi.userAgent], [Cgi.referrer], [Cgi.accept], [Cgi.authorization], [Cgi.lastEventId]) 235 236 Output: [Cgi.write], [Cgi.header], [Cgi.setResponseStatus], [Cgi.setResponseContentType], [Cgi.gzipResponse] 237 238 Cookies: [Cgi.setCookie], [Cgi.clearCookie], [Cgi.cookie], [Cgi.cookies] 239 240 Caching: [Cgi.setResponseExpires], [Cgi.updateResponseExpires], [Cgi.setCache] 241 242 Redirections: [Cgi.setResponseLocation] 243 244 Other Information: [Cgi.remoteAddress], [Cgi.https], [Cgi.port], [Cgi.scriptName], [Cgi.requestUri], [Cgi.getCurrentCompleteUri], [Cgi.onRequestBodyDataReceived] 245 246 Overriding behavior: [Cgi.handleIncomingDataChunk], [Cgi.prepareForIncomingDataChunks], [Cgi.cleanUpPostDataState] 247 248 Installing: Apache, IIS, CGI, FastCGI, SCGI, embedded HTTPD (not recommended for production use) 249 250 Guide_for_PHP_users: 251 If you are coming from PHP, here's a quick guide to help you get started: 252 253 $(SIDE_BY_SIDE 254 $(COLUMN 255 ```php 256 <?php 257 $foo = $_POST["foo"]; 258 $bar = $_GET["bar"]; 259 $baz = $_COOKIE["baz"]; 260 261 $user_ip = $_SERVER["REMOTE_ADDR"]; 262 $host = $_SERVER["HTTP_HOST"]; 263 $path = $_SERVER["PATH_INFO"]; 264 265 setcookie("baz", "some value"); 266 267 echo "hello!"; 268 ?> 269 ``` 270 ) 271 $(COLUMN 272 --- 273 import arsd.cgi; 274 void app(Cgi cgi) { 275 string foo = cgi.post["foo"]; 276 string bar = cgi.get["bar"]; 277 string baz = cgi.cookies["baz"]; 278 279 string user_ip = cgi.remoteAddress; 280 string host = cgi.host; 281 string path = cgi.pathInfo; 282 283 cgi.setCookie("baz", "some value"); 284 285 cgi.write("hello!"); 286 } 287 288 mixin GenericMain!app 289 --- 290 ) 291 ) 292 293 $(H3 Array elements) 294 295 296 In PHP, you can give a form element a name like `"something[]"`, and then 297 `$_POST["something"]` gives an array. In D, you can use whatever name 298 you want, and access an array of values with the `cgi.getArray["name"]` and 299 `cgi.postArray["name"]` members. 300 301 $(H3 Databases) 302 303 PHP has a lot of stuff in its standard library. cgi.d doesn't include most 304 of these, but the rest of my arsd repository has much of it. For example, 305 to access a MySQL database, download `database.d` and `mysql.d` from my 306 github repo, and try this code (assuming, of course, your database is 307 set up): 308 309 --- 310 import arsd.cgi; 311 import arsd.mysql; 312 313 void app(Cgi cgi) { 314 auto database = new MySql("localhost", "username", "password", "database_name"); 315 foreach(row; mysql.query("SELECT count(id) FROM people")) 316 cgi.write(row[0] ~ " people in database"); 317 } 318 319 mixin GenericMain!app; 320 --- 321 322 Similar modules are available for PostgreSQL, Microsoft SQL Server, and SQLite databases, 323 implementing the same basic interface. 324 325 See_Also: 326 327 You may also want to see [arsd.dom], [arsd.web], and [arsd.html] for more code for making 328 web applications. 329 330 For working with json, try [arsd.jsvar]. 331 332 [arsd.database], [arsd.mysql], [arsd.postgres], [arsd.mssql], and [arsd.sqlite] can help in 333 accessing databases. 334 335 If you are looking to access a web application via HTTP, try [std.net.curl], [arsd.curl], or [arsd.http2]. 336 337 Copyright: 338 339 cgi.d copyright 2008-2021, Adam D. Ruppe. Provided under the Boost Software License. 340 341 Yes, this file is old, and yes, it is still actively maintained and used. 342 +/ 343 module arsd.cgi; 344 345 version(Demo) 346 unittest { 347 348 } 349 350 static import std.file; 351 352 // for a single thread, linear request thing, use: 353 // -version=embedded_httpd_threads -version=cgi_no_threads 354 355 version(Posix) { 356 version(CRuntime_Musl) { 357 358 } else version(minimal) { 359 360 } else { 361 version(GNU) { 362 // GDC doesn't support static foreach so I had to cheat on it :( 363 } else version(FreeBSD) { 364 // I never implemented the fancy stuff there either 365 } else { 366 version=with_breaking_cgi_features; 367 version=with_sendfd; 368 version=with_addon_servers; 369 } 370 } 371 } 372 373 void cloexec(int fd) { 374 version(Posix) { 375 import core.sys.posix.fcntl; 376 fcntl(fd, F_SETFD, FD_CLOEXEC); 377 } 378 } 379 380 void cloexec(Socket s) { 381 version(Posix) { 382 import core.sys.posix.fcntl; 383 fcntl(s.handle, F_SETFD, FD_CLOEXEC); 384 } 385 } 386 387 version(embedded_httpd_hybrid) { 388 version=embedded_httpd_threads; 389 version(cgi_no_fork) {} else 390 version=cgi_use_fork; 391 version=cgi_use_fiber; 392 } 393 394 // the servers must know about the connections to talk to them; the interfaces are vital 395 version(with_addon_servers) 396 version=with_addon_servers_connections; 397 398 version(embedded_httpd) { 399 version(linux) 400 version=embedded_httpd_processes; 401 else { 402 version=embedded_httpd_threads; 403 } 404 405 /* 406 version(with_openssl) { 407 pragma(lib, "crypto"); 408 pragma(lib, "ssl"); 409 } 410 */ 411 } 412 413 version(embedded_httpd_processes) 414 version=embedded_httpd_processes_accept_after_fork; // I am getting much better average performance on this, so just keeping it. But the other way MIGHT help keep the variation down so i wanna keep the code to play with later 415 416 version(embedded_httpd_threads) { 417 // unless the user overrides the default.. 418 version(cgi_session_server_process) 419 {} 420 else 421 version=cgi_embedded_sessions; 422 } 423 version(scgi) { 424 // unless the user overrides the default.. 425 version(cgi_session_server_process) 426 {} 427 else 428 version=cgi_embedded_sessions; 429 } 430 431 // fall back if the other is not defined so we can cleanly version it below 432 version(cgi_embedded_sessions) {} 433 else version=cgi_session_server_process; 434 435 436 version=cgi_with_websocket; 437 438 enum long defaultMaxContentLength = 5_000_000; 439 440 /* 441 442 To do a file download offer in the browser: 443 444 cgi.setResponseContentType("text/csv"); 445 cgi.header("Content-Disposition: attachment; filename=\"customers.csv\""); 446 */ 447 448 // FIXME: the location header is supposed to be an absolute url I guess. 449 450 // FIXME: would be cool to flush part of a dom document before complete 451 // somehow in here and dom.d. 452 453 454 // these are public so you can mixin GenericMain. 455 // FIXME: use a function level import instead! 456 public import std.string; 457 public import std.stdio; 458 public import std.conv; 459 import std.uri; 460 import std.uni; 461 import std.algorithm.comparison; 462 import std.algorithm.searching; 463 import std.exception; 464 import std.base64; 465 static import std.algorithm; 466 import std.datetime; 467 import std.range; 468 469 import std.process; 470 471 import std.zlib; 472 473 474 T[] consume(T)(T[] range, int count) { 475 if(count > range.length) 476 count = range.length; 477 return range[count..$]; 478 } 479 480 int locationOf(T)(T[] data, string item) { 481 const(ubyte[]) d = cast(const(ubyte[])) data; 482 const(ubyte[]) i = cast(const(ubyte[])) item; 483 484 // this is a vague sanity check to ensure we aren't getting insanely 485 // sized input that will infinite loop below. it should never happen; 486 // even huge file uploads ought to come in smaller individual pieces. 487 if(d.length > (int.max/2)) 488 throw new Exception("excessive block of input"); 489 490 for(int a = 0; a < d.length; a++) { 491 if(a + i.length > d.length) 492 return -1; 493 if(d[a..a+i.length] == i) 494 return a; 495 } 496 497 return -1; 498 } 499 500 /// If you are doing a custom cgi class, mixing this in can take care of 501 /// the required constructors for you 502 mixin template ForwardCgiConstructors() { 503 this(long maxContentLength = defaultMaxContentLength, 504 string[string] env = null, 505 const(ubyte)[] delegate() readdata = null, 506 void delegate(const(ubyte)[]) _rawDataOutput = null, 507 void delegate() _flush = null 508 ) { super(maxContentLength, env, readdata, _rawDataOutput, _flush); } 509 510 this(string[] args) { super(args); } 511 512 this( 513 BufferedInputRange inputData, 514 string address, ushort _port, 515 int pathInfoStarts = 0, 516 bool _https = false, 517 void delegate(const(ubyte)[]) _rawDataOutput = null, 518 void delegate() _flush = null, 519 // this pointer tells if the connection is supposed to be closed after we handle this 520 bool* closeConnection = null) 521 { 522 super(inputData, address, _port, pathInfoStarts, _https, _rawDataOutput, _flush, closeConnection); 523 } 524 525 this(BufferedInputRange ir, bool* closeConnection) { super(ir, closeConnection); } 526 } 527 528 /// thrown when a connection is closed remotely while we waiting on data from it 529 class ConnectionClosedException : Exception { 530 this(string message, string file = __FILE__, size_t line = __LINE__, Throwable next = null) { 531 super(message, file, line, next); 532 } 533 } 534 535 536 version(Windows) { 537 // FIXME: ugly hack to solve stdin exception problems on Windows: 538 // reading stdin results in StdioException (Bad file descriptor) 539 // this is probably due to http://d.puremagic.com/issues/show_bug.cgi?id=3425 540 private struct stdin { 541 struct ByChunk { // Replicates std.stdio.ByChunk 542 private: 543 ubyte[] chunk_; 544 public: 545 this(size_t size) 546 in { 547 assert(size, "size must be larger than 0"); 548 } 549 do { 550 chunk_ = new ubyte[](size); 551 popFront(); 552 } 553 554 @property bool empty() const { 555 return !std.stdio.stdin.isOpen || std.stdio.stdin.eof; // Ugly, but seems to do the job 556 } 557 @property nothrow ubyte[] front() { return chunk_; } 558 void popFront() { 559 enforce(!empty, "Cannot call popFront on empty range"); 560 chunk_ = stdin.rawRead(chunk_); 561 } 562 } 563 564 import core.sys.windows.windows; 565 static: 566 567 static this() { 568 // Set stdin to binary mode 569 version(Win64) 570 _setmode(std.stdio.stdin.fileno(), 0x8000); 571 else 572 setmode(std.stdio.stdin.fileno(), 0x8000); 573 } 574 575 T[] rawRead(T)(T[] buf) { 576 uint bytesRead; 577 auto result = ReadFile(GetStdHandle(STD_INPUT_HANDLE), buf.ptr, cast(int) (buf.length * T.sizeof), &bytesRead, null); 578 579 if (!result) { 580 auto err = GetLastError(); 581 if (err == 38/*ERROR_HANDLE_EOF*/ || err == 109/*ERROR_BROKEN_PIPE*/) // 'good' errors meaning end of input 582 return buf[0..0]; 583 // Some other error, throw it 584 585 char* buffer; 586 scope(exit) LocalFree(buffer); 587 588 // FORMAT_MESSAGE_ALLOCATE_BUFFER = 0x00000100 589 // FORMAT_MESSAGE_FROM_SYSTEM = 0x00001000 590 FormatMessageA(0x1100, null, err, 0, cast(char*)&buffer, 256, null); 591 throw new Exception(to!string(buffer)); 592 } 593 enforce(!(bytesRead % T.sizeof), "I/O error"); 594 return buf[0..bytesRead / T.sizeof]; 595 } 596 597 auto byChunk(size_t sz) { return ByChunk(sz); } 598 599 void close() { 600 std.stdio.stdin.close; 601 } 602 } 603 } 604 605 /// The main interface with the web request 606 class Cgi { 607 public: 608 /// the methods a request can be 609 enum RequestMethod { GET, HEAD, POST, PUT, DELETE, // GET and POST are the ones that really work 610 // these are defined in the standard, but idk if they are useful for anything 611 OPTIONS, TRACE, CONNECT, 612 // These seem new, I have only recently seen them 613 PATCH, MERGE, 614 // this is an extension for when the method is not specified and you want to assume 615 CommandLine } 616 617 618 /+ 619 /++ 620 Cgi provides a per-request memory pool 621 622 +/ 623 void[] allocateMemory(size_t nBytes) { 624 625 } 626 627 /// ditto 628 void[] reallocateMemory(void[] old, size_t nBytes) { 629 630 } 631 632 /// ditto 633 void freeMemory(void[] memory) { 634 635 } 636 +/ 637 638 639 /* 640 import core.runtime; 641 auto args = Runtime.args(); 642 643 we can call the app a few ways: 644 645 1) set up the environment variables and call the app (manually simulating CGI) 646 2) simulate a call automatically: 647 ./app method 'uri' 648 649 for example: 650 ./app get /path?arg arg2=something 651 652 Anything on the uri is treated as query string etc 653 654 on get method, further args are appended to the query string (encoded automatically) 655 on post method, further args are done as post 656 657 658 @name means import from file "name". if name == -, it uses stdin 659 (so info=@- means set info to the value of stdin) 660 661 662 Other arguments include: 663 --cookie name=value (these are all concated together) 664 --header 'X-Something: cool' 665 --referrer 'something' 666 --port 80 667 --remote-address some.ip.address.here 668 --https yes 669 --user-agent 'something' 670 --userpass 'user:pass' 671 --authorization 'Basic base64encoded_user:pass' 672 --accept 'content' // FIXME: better example 673 --last-event-id 'something' 674 --host 'something.com' 675 676 Non-simulation arguments: 677 --port xxx listening port for non-cgi things (valid for the cgi interfaces) 678 --listening-host the ip address the application should listen on, or if you want to use unix domain sockets, it is here you can set them: `--listening-host unix:filename` or, on Linux, `--listening-host abstract:name`. 679 680 */ 681 682 /** Initializes it with command line arguments (for easy testing) */ 683 this(string[] args, void delegate(const(ubyte)[]) _rawDataOutput = null) { 684 rawDataOutput = _rawDataOutput; 685 // these are all set locally so the loop works 686 // without triggering errors in dmd 2.064 687 // we go ahead and set them at the end of it to the this version 688 int port; 689 string referrer; 690 string remoteAddress; 691 string userAgent; 692 string authorization; 693 string origin; 694 string accept; 695 string lastEventId; 696 bool https; 697 string host; 698 RequestMethod requestMethod; 699 string requestUri; 700 string pathInfo; 701 string queryString; 702 703 bool lookingForMethod; 704 bool lookingForUri; 705 string nextArgIs; 706 707 string _cookie; 708 string _queryString; 709 string[][string] _post; 710 string[string] _headers; 711 712 string[] breakUp(string s) { 713 string k, v; 714 auto idx = s.indexOf("="); 715 if(idx == -1) { 716 k = s; 717 } else { 718 k = s[0 .. idx]; 719 v = s[idx + 1 .. $]; 720 } 721 722 return [k, v]; 723 } 724 725 lookingForMethod = true; 726 727 scriptName = args[0]; 728 scriptFileName = args[0]; 729 730 environmentVariables = cast(const) environment.toAA; 731 732 foreach(arg; args[1 .. $]) { 733 if(arg.startsWith("--")) { 734 nextArgIs = arg[2 .. $]; 735 } else if(nextArgIs.length) { 736 if (nextArgIs == "cookie") { 737 auto info = breakUp(arg); 738 if(_cookie.length) 739 _cookie ~= "; "; 740 _cookie ~= std.uri.encodeComponent(info[0]) ~ "=" ~ std.uri.encodeComponent(info[1]); 741 } 742 else if (nextArgIs == "port") { 743 port = to!int(arg); 744 } 745 else if (nextArgIs == "referrer") { 746 referrer = arg; 747 } 748 else if (nextArgIs == "remote-address") { 749 remoteAddress = arg; 750 } 751 else if (nextArgIs == "user-agent") { 752 userAgent = arg; 753 } 754 else if (nextArgIs == "authorization") { 755 authorization = arg; 756 } 757 else if (nextArgIs == "userpass") { 758 authorization = "Basic " ~ Base64.encode(cast(immutable(ubyte)[]) (arg)).idup; 759 } 760 else if (nextArgIs == "origin") { 761 origin = arg; 762 } 763 else if (nextArgIs == "accept") { 764 accept = arg; 765 } 766 else if (nextArgIs == "last-event-id") { 767 lastEventId = arg; 768 } 769 else if (nextArgIs == "https") { 770 if(arg == "yes") 771 https = true; 772 } 773 else if (nextArgIs == "header") { 774 string thing, other; 775 auto idx = arg.indexOf(":"); 776 if(idx == -1) 777 throw new Exception("need a colon in a http header"); 778 thing = arg[0 .. idx]; 779 other = arg[idx + 1.. $]; 780 _headers[thing.strip.toLower()] = other.strip; 781 } 782 else if (nextArgIs == "host") { 783 host = arg; 784 } 785 // else 786 // skip, we don't know it but that's ok, it might be used elsewhere so no error 787 788 nextArgIs = null; 789 } else if(lookingForMethod) { 790 lookingForMethod = false; 791 lookingForUri = true; 792 793 if(arg.asLowerCase().equal("commandline")) 794 requestMethod = RequestMethod.CommandLine; 795 else 796 requestMethod = to!RequestMethod(arg.toUpper()); 797 } else if(lookingForUri) { 798 lookingForUri = false; 799 800 requestUri = arg; 801 802 auto idx = arg.indexOf("?"); 803 if(idx == -1) 804 pathInfo = arg; 805 else { 806 pathInfo = arg[0 .. idx]; 807 _queryString = arg[idx + 1 .. $]; 808 } 809 } else { 810 // it is an argument of some sort 811 if(requestMethod == Cgi.RequestMethod.POST || requestMethod == Cgi.RequestMethod.PATCH || requestMethod == Cgi.RequestMethod.PUT || requestMethod == Cgi.RequestMethod.CommandLine) { 812 auto parts = breakUp(arg); 813 _post[parts[0]] ~= parts[1]; 814 allPostNamesInOrder ~= parts[0]; 815 allPostValuesInOrder ~= parts[1]; 816 } else { 817 if(_queryString.length) 818 _queryString ~= "&"; 819 auto parts = breakUp(arg); 820 _queryString ~= std.uri.encodeComponent(parts[0]) ~ "=" ~ std.uri.encodeComponent(parts[1]); 821 } 822 } 823 } 824 825 acceptsGzip = false; 826 keepAliveRequested = false; 827 requestHeaders = cast(immutable) _headers; 828 829 cookie = _cookie; 830 cookiesArray = getCookieArray(); 831 cookies = keepLastOf(cookiesArray); 832 833 queryString = _queryString; 834 getArray = cast(immutable) decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 835 get = keepLastOf(getArray); 836 837 postArray = cast(immutable) _post; 838 post = keepLastOf(_post); 839 840 // FIXME 841 filesArray = null; 842 files = null; 843 844 isCalledWithCommandLineArguments = true; 845 846 this.port = port; 847 this.referrer = referrer; 848 this.remoteAddress = remoteAddress; 849 this.userAgent = userAgent; 850 this.authorization = authorization; 851 this.origin = origin; 852 this.accept = accept; 853 this.lastEventId = lastEventId; 854 this.https = https; 855 this.host = host; 856 this.requestMethod = requestMethod; 857 this.requestUri = requestUri; 858 this.pathInfo = pathInfo; 859 this.queryString = queryString; 860 this.postBody = null; 861 } 862 863 private { 864 string[] allPostNamesInOrder; 865 string[] allPostValuesInOrder; 866 string[] allGetNamesInOrder; 867 string[] allGetValuesInOrder; 868 } 869 870 CgiConnectionHandle getOutputFileHandle() { 871 return _outputFileHandle; 872 } 873 874 CgiConnectionHandle _outputFileHandle = INVALID_CGI_CONNECTION_HANDLE; 875 876 /** Initializes it using a CGI or CGI-like interface */ 877 this(long maxContentLength = defaultMaxContentLength, 878 // use this to override the environment variable listing 879 in string[string] env = null, 880 // and this should return a chunk of data. return empty when done 881 const(ubyte)[] delegate() readdata = null, 882 // finally, use this to do custom output if needed 883 void delegate(const(ubyte)[]) _rawDataOutput = null, 884 // to flush teh custom output 885 void delegate() _flush = null 886 ) 887 { 888 889 // these are all set locally so the loop works 890 // without triggering errors in dmd 2.064 891 // we go ahead and set them at the end of it to the this version 892 int port; 893 string referrer; 894 string remoteAddress; 895 string userAgent; 896 string authorization; 897 string origin; 898 string accept; 899 string lastEventId; 900 bool https; 901 string host; 902 RequestMethod requestMethod; 903 string requestUri; 904 string pathInfo; 905 string queryString; 906 907 908 909 isCalledWithCommandLineArguments = false; 910 rawDataOutput = _rawDataOutput; 911 flushDelegate = _flush; 912 auto getenv = delegate string(string var) { 913 if(env is null) 914 return std.process.environment.get(var); 915 auto e = var in env; 916 if(e is null) 917 return null; 918 return *e; 919 }; 920 921 environmentVariables = env is null ? 922 cast(const) environment.toAA : 923 env; 924 925 // fetching all the request headers 926 string[string] requestHeadersHere; 927 foreach(k, v; env is null ? cast(const) environment.toAA() : env) { 928 if(k.startsWith("HTTP_")) { 929 requestHeadersHere[replace(k["HTTP_".length .. $].toLower(), "_", "-")] = v; 930 } 931 } 932 933 this.requestHeaders = assumeUnique(requestHeadersHere); 934 935 requestUri = getenv("REQUEST_URI"); 936 937 cookie = getenv("HTTP_COOKIE"); 938 cookiesArray = getCookieArray(); 939 cookies = keepLastOf(cookiesArray); 940 941 referrer = getenv("HTTP_REFERER"); 942 userAgent = getenv("HTTP_USER_AGENT"); 943 remoteAddress = getenv("REMOTE_ADDR"); 944 host = getenv("HTTP_HOST"); 945 pathInfo = getenv("PATH_INFO"); 946 947 queryString = getenv("QUERY_STRING"); 948 scriptName = getenv("SCRIPT_NAME"); 949 { 950 import core.runtime; 951 auto sfn = getenv("SCRIPT_FILENAME"); 952 scriptFileName = sfn.length ? sfn : Runtime.args[0]; 953 } 954 955 bool iis = false; 956 957 // Because IIS doesn't pass requestUri, we simulate it here if it's empty. 958 if(requestUri.length == 0) { 959 // IIS sometimes includes the script name as part of the path info - we don't want that 960 if(pathInfo.length >= scriptName.length && (pathInfo[0 .. scriptName.length] == scriptName)) 961 pathInfo = pathInfo[scriptName.length .. $]; 962 963 requestUri = scriptName ~ pathInfo ~ (queryString.length ? ("?" ~ queryString) : ""); 964 965 iis = true; // FIXME HACK - used in byChunk below - see bugzilla 6339 966 967 // FIXME: this works for apache and iis... but what about others? 968 } 969 970 971 auto ugh = decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 972 getArray = assumeUnique(ugh); 973 get = keepLastOf(getArray); 974 975 976 // NOTE: on shitpache, you need to specifically forward this 977 authorization = getenv("HTTP_AUTHORIZATION"); 978 // this is a hack because Apache is a shitload of fuck and 979 // refuses to send the real header to us. Compatible 980 // programs should send both the standard and X- versions 981 982 // NOTE: if you have access to .htaccess or httpd.conf, you can make this 983 // unnecessary with mod_rewrite, so it is commented 984 985 //if(authorization.length == 0) // if the std is there, use it 986 // authorization = getenv("HTTP_X_AUTHORIZATION"); 987 988 // the REDIRECT_HTTPS check is here because with an Apache hack, the port can become wrong 989 if(getenv("SERVER_PORT").length && getenv("REDIRECT_HTTPS") != "on") 990 port = to!int(getenv("SERVER_PORT")); 991 else 992 port = 0; // this was probably called from the command line 993 994 auto ae = getenv("HTTP_ACCEPT_ENCODING"); 995 if(ae.length && ae.indexOf("gzip") != -1) 996 acceptsGzip = true; 997 998 accept = getenv("HTTP_ACCEPT"); 999 lastEventId = getenv("HTTP_LAST_EVENT_ID"); 1000 1001 auto ka = getenv("HTTP_CONNECTION"); 1002 if(ka.length && ka.asLowerCase().canFind("keep-alive")) 1003 keepAliveRequested = true; 1004 1005 auto or = getenv("HTTP_ORIGIN"); 1006 origin = or; 1007 1008 auto rm = getenv("REQUEST_METHOD"); 1009 if(rm.length) 1010 requestMethod = to!RequestMethod(getenv("REQUEST_METHOD")); 1011 else 1012 requestMethod = RequestMethod.CommandLine; 1013 1014 // FIXME: hack on REDIRECT_HTTPS; this is there because the work app uses mod_rewrite which loses the https flag! So I set it with [E=HTTPS=%HTTPS] or whatever but then it gets translated to here so i want it to still work. This is arguably wrong but meh. 1015 https = (getenv("HTTPS") == "on" || getenv("REDIRECT_HTTPS") == "on"); 1016 1017 // FIXME: DOCUMENT_ROOT? 1018 1019 // FIXME: what about PUT? 1020 if(requestMethod == RequestMethod.POST || requestMethod == Cgi.RequestMethod.PATCH || requestMethod == Cgi.RequestMethod.PUT || requestMethod == Cgi.RequestMethod.CommandLine) { 1021 version(preserveData) // a hack to make forwarding simpler 1022 immutable(ubyte)[] data; 1023 size_t amountReceived = 0; 1024 auto contentType = getenv("CONTENT_TYPE"); 1025 1026 // FIXME: is this ever not going to be set? I guess it depends 1027 // on if the server de-chunks and buffers... seems like it has potential 1028 // to be slow if they did that. The spec says it is always there though. 1029 // And it has worked reliably for me all year in the live environment, 1030 // but some servers might be different. 1031 auto contentLength = to!size_t(getenv("CONTENT_LENGTH")); 1032 1033 immutable originalContentLength = contentLength; 1034 if(contentLength) { 1035 if(maxContentLength > 0 && contentLength > maxContentLength) { 1036 setResponseStatus("413 Request entity too large"); 1037 write("You tried to upload a file that is too large."); 1038 close(); 1039 throw new Exception("POST too large"); 1040 } 1041 prepareForIncomingDataChunks(contentType, contentLength); 1042 1043 1044 int processChunk(in ubyte[] chunk) { 1045 if(chunk.length > contentLength) { 1046 handleIncomingDataChunk(chunk[0..contentLength]); 1047 amountReceived += contentLength; 1048 contentLength = 0; 1049 return 1; 1050 } else { 1051 handleIncomingDataChunk(chunk); 1052 contentLength -= chunk.length; 1053 amountReceived += chunk.length; 1054 } 1055 if(contentLength == 0) 1056 return 1; 1057 1058 onRequestBodyDataReceived(amountReceived, originalContentLength); 1059 return 0; 1060 } 1061 1062 1063 if(readdata is null) { 1064 foreach(ubyte[] chunk; stdin.byChunk(iis ? contentLength : 4096)) 1065 if(processChunk(chunk)) 1066 break; 1067 } else { 1068 // we have a custom data source.. 1069 auto chunk = readdata(); 1070 while(chunk.length) { 1071 if(processChunk(chunk)) 1072 break; 1073 chunk = readdata(); 1074 } 1075 } 1076 1077 onRequestBodyDataReceived(amountReceived, originalContentLength); 1078 postArray = assumeUnique(pps._post); 1079 filesArray = assumeUnique(pps._files); 1080 files = keepLastOf(filesArray); 1081 post = keepLastOf(postArray); 1082 this.postBody = pps.postBody; 1083 cleanUpPostDataState(); 1084 } 1085 1086 version(preserveData) 1087 originalPostData = data; 1088 } 1089 // fixme: remote_user script name 1090 1091 1092 this.port = port; 1093 this.referrer = referrer; 1094 this.remoteAddress = remoteAddress; 1095 this.userAgent = userAgent; 1096 this.authorization = authorization; 1097 this.origin = origin; 1098 this.accept = accept; 1099 this.lastEventId = lastEventId; 1100 this.https = https; 1101 this.host = host; 1102 this.requestMethod = requestMethod; 1103 this.requestUri = requestUri; 1104 this.pathInfo = pathInfo; 1105 this.queryString = queryString; 1106 } 1107 1108 /// Cleans up any temporary files. Do not use the object 1109 /// after calling this. 1110 /// 1111 /// NOTE: it is called automatically by GenericMain 1112 // FIXME: this should be called if the constructor fails too, if it has created some garbage... 1113 void dispose() { 1114 foreach(file; files) { 1115 if(!file.contentInMemory) 1116 if(std.file.exists(file.contentFilename)) 1117 std.file.remove(file.contentFilename); 1118 } 1119 } 1120 1121 private { 1122 struct PostParserState { 1123 string contentType; 1124 string boundary; 1125 string localBoundary; // the ones used at the end or something lol 1126 bool isMultipart; 1127 bool needsSavedBody; 1128 1129 ulong expectedLength; 1130 ulong contentConsumed; 1131 immutable(ubyte)[] buffer; 1132 1133 // multipart parsing state 1134 int whatDoWeWant; 1135 bool weHaveAPart; 1136 string[] thisOnesHeaders; 1137 immutable(ubyte)[] thisOnesData; 1138 1139 string postBody; 1140 1141 UploadedFile piece; 1142 bool isFile = false; 1143 1144 size_t memoryCommitted; 1145 1146 // do NOT keep mutable references to these anywhere! 1147 // I assume they are unique in the constructor once we're all done getting data. 1148 string[][string] _post; 1149 UploadedFile[][string] _files; 1150 } 1151 1152 PostParserState pps; 1153 } 1154 1155 /// This represents a file the user uploaded via a POST request. 1156 static struct UploadedFile { 1157 /// If you want to create one of these structs for yourself from some data, 1158 /// use this function. 1159 static UploadedFile fromData(immutable(void)[] data, string name = null) { 1160 Cgi.UploadedFile f; 1161 f.filename = name; 1162 f.content = cast(immutable(ubyte)[]) data; 1163 f.contentInMemory = true; 1164 return f; 1165 } 1166 1167 string name; /// The name of the form element. 1168 string filename; /// The filename the user set. 1169 string contentType; /// The MIME type the user's browser reported. (Not reliable.) 1170 1171 /** 1172 For small files, cgi.d will buffer the uploaded file in memory, and make it 1173 directly accessible to you through the content member. I find this very convenient 1174 and somewhat efficient, since it can avoid hitting the disk entirely. (I 1175 often want to inspect and modify the file anyway!) 1176 1177 I find the file is very large, it is undesirable to eat that much memory just 1178 for a file buffer. In those cases, if you pass a large enough value for maxContentLength 1179 to the constructor so they are accepted, cgi.d will write the content to a temporary 1180 file that you can re-read later. 1181 1182 You can override this behavior by subclassing Cgi and overriding the protected 1183 handlePostChunk method. Note that the object is not initialized when you 1184 write that method - the http headers are available, but the cgi.post method 1185 is not. You may parse the file as it streams in using this method. 1186 1187 1188 Anyway, if the file is small enough to be in memory, contentInMemory will be 1189 set to true, and the content is available in the content member. 1190 1191 If not, contentInMemory will be set to false, and the content saved in a file, 1192 whose name will be available in the contentFilename member. 1193 1194 1195 Tip: if you know you are always dealing with small files, and want the convenience 1196 of ignoring this member, construct Cgi with a small maxContentLength. Then, if 1197 a large file comes in, it simply throws an exception (and HTTP error response) 1198 instead of trying to handle it. 1199 1200 The default value of maxContentLength in the constructor is for small files. 1201 */ 1202 bool contentInMemory = true; // the default ought to always be true 1203 immutable(ubyte)[] content; /// The actual content of the file, if contentInMemory == true 1204 string contentFilename; /// the file where we dumped the content, if contentInMemory == false. Note that if you want to keep it, you MUST move the file, since otherwise it is considered garbage when cgi is disposed. 1205 1206 /// 1207 ulong fileSize() { 1208 if(contentInMemory) 1209 return content.length; 1210 import std.file; 1211 return std.file.getSize(contentFilename); 1212 1213 } 1214 1215 /// 1216 void writeToFile(string filenameToSaveTo) const { 1217 import std.file; 1218 if(contentInMemory) 1219 std.file.write(filenameToSaveTo, content); 1220 else 1221 std.file.rename(contentFilename, filenameToSaveTo); 1222 } 1223 } 1224 1225 // given a content type and length, decide what we're going to do with the data.. 1226 protected void prepareForIncomingDataChunks(string contentType, ulong contentLength) { 1227 pps.expectedLength = contentLength; 1228 1229 auto terminator = contentType.indexOf(";"); 1230 if(terminator == -1) 1231 terminator = contentType.length; 1232 1233 pps.contentType = contentType[0 .. terminator]; 1234 auto b = contentType[terminator .. $]; 1235 if(b.length) { 1236 auto idx = b.indexOf("boundary="); 1237 if(idx != -1) { 1238 pps.boundary = b[idx + "boundary=".length .. $]; 1239 pps.localBoundary = "\r\n--" ~ pps.boundary; 1240 } 1241 } 1242 1243 // while a content type SHOULD be sent according to the RFC, it is 1244 // not required. We're told we SHOULD guess by looking at the content 1245 // but it seems to me that this only happens when it is urlencoded. 1246 if(pps.contentType == "application/x-www-form-urlencoded" || pps.contentType == "") { 1247 pps.isMultipart = false; 1248 pps.needsSavedBody = false; 1249 } else if(pps.contentType == "multipart/form-data") { 1250 pps.isMultipart = true; 1251 enforce(pps.boundary.length, "no boundary"); 1252 } else if(pps.contentType == "text/xml") { // FIXME: could this be special and load the post params 1253 // save the body so the application can handle it 1254 pps.isMultipart = false; 1255 pps.needsSavedBody = true; 1256 } else if(pps.contentType == "application/json") { // FIXME: this could prolly try to load post params too 1257 // save the body so the application can handle it 1258 pps.needsSavedBody = true; 1259 pps.isMultipart = false; 1260 } else { 1261 // the rest is 100% handled by the application. just save the body and send it to them 1262 pps.needsSavedBody = true; 1263 pps.isMultipart = false; 1264 } 1265 } 1266 1267 // handles streaming POST data. If you handle some other content type, you should 1268 // override this. If the data isn't the content type you want, you ought to call 1269 // super.handleIncomingDataChunk so regular forms and files still work. 1270 1271 // FIXME: I do some copying in here that I'm pretty sure is unnecessary, and the 1272 // file stuff I'm sure is inefficient. But, my guess is the real bottleneck is network 1273 // input anyway, so I'm not going to get too worked up about it right now. 1274 protected void handleIncomingDataChunk(const(ubyte)[] chunk) { 1275 if(chunk.length == 0) 1276 return; 1277 assert(chunk.length <= 32 * 1024 * 1024); // we use chunk size as a memory constraint thing, so 1278 // if we're passed big chunks, it might throw unnecessarily. 1279 // just pass it smaller chunks at a time. 1280 if(pps.isMultipart) { 1281 // multipart/form-data 1282 1283 1284 // FIXME: this might want to be factored out and factorized 1285 // need to make sure the stream hooks actually work. 1286 void pieceHasNewContent() { 1287 // we just grew the piece's buffer. Do we have to switch to file backing? 1288 if(pps.piece.contentInMemory) { 1289 if(pps.piece.content.length <= 10 * 1024 * 1024) 1290 // meh, I'm ok with it. 1291 return; 1292 else { 1293 // this is too big. 1294 if(!pps.isFile) 1295 throw new Exception("Request entity too large"); // a variable this big is kinda ridiculous, just reject it. 1296 else { 1297 // a file this large is probably acceptable though... let's use a backing file. 1298 pps.piece.contentInMemory = false; 1299 // FIXME: say... how do we intend to delete these things? cgi.dispose perhaps. 1300 1301 int count = 0; 1302 pps.piece.contentFilename = getTempDirectory() ~ "arsd_cgi_uploaded_file_" ~ to!string(getUtcTime()) ~ "-" ~ to!string(count); 1303 // odds are this loop will never be entered, but we want it just in case. 1304 while(std.file.exists(pps.piece.contentFilename)) { 1305 count++; 1306 pps.piece.contentFilename = getTempDirectory() ~ "arsd_cgi_uploaded_file_" ~ to!string(getUtcTime()) ~ "-" ~ to!string(count); 1307 } 1308 // I hope this creates the file pretty quickly, or the loop might be useless... 1309 // FIXME: maybe I should write some kind of custom transaction here. 1310 std.file.write(pps.piece.contentFilename, pps.piece.content); 1311 1312 pps.piece.content = null; 1313 } 1314 } 1315 } else { 1316 // it's already in a file, so just append it to what we have 1317 if(pps.piece.content.length) { 1318 // FIXME: this is surely very inefficient... we'll be calling this by 4kb chunk... 1319 std.file.append(pps.piece.contentFilename, pps.piece.content); 1320 pps.piece.content = null; 1321 } 1322 } 1323 } 1324 1325 1326 void commitPart() { 1327 if(!pps.weHaveAPart) 1328 return; 1329 1330 pieceHasNewContent(); // be sure the new content is handled every time 1331 1332 if(pps.isFile) { 1333 // I'm not sure if other environments put files in post or not... 1334 // I used to not do it, but I think I should, since it is there... 1335 pps._post[pps.piece.name] ~= pps.piece.filename; 1336 pps._files[pps.piece.name] ~= pps.piece; 1337 1338 allPostNamesInOrder ~= pps.piece.name; 1339 allPostValuesInOrder ~= pps.piece.filename; 1340 } else { 1341 pps._post[pps.piece.name] ~= cast(string) pps.piece.content; 1342 1343 allPostNamesInOrder ~= pps.piece.name; 1344 allPostValuesInOrder ~= cast(string) pps.piece.content; 1345 } 1346 1347 /* 1348 stderr.writeln("RECEIVED: ", pps.piece.name, "=", 1349 pps.piece.content.length < 1000 1350 ? 1351 to!string(pps.piece.content) 1352 : 1353 "too long"); 1354 */ 1355 1356 // FIXME: the limit here 1357 pps.memoryCommitted += pps.piece.content.length; 1358 1359 pps.weHaveAPart = false; 1360 pps.whatDoWeWant = 1; 1361 pps.thisOnesHeaders = null; 1362 pps.thisOnesData = null; 1363 1364 pps.piece = UploadedFile.init; 1365 pps.isFile = false; 1366 } 1367 1368 void acceptChunk() { 1369 pps.buffer ~= chunk; 1370 chunk = null; // we've consumed it into the buffer, so keeping it just brings confusion 1371 } 1372 1373 immutable(ubyte)[] consume(size_t howMuch) { 1374 pps.contentConsumed += howMuch; 1375 auto ret = pps.buffer[0 .. howMuch]; 1376 pps.buffer = pps.buffer[howMuch .. $]; 1377 return ret; 1378 } 1379 1380 dataConsumptionLoop: do { 1381 switch(pps.whatDoWeWant) { 1382 default: assert(0); 1383 case 0: 1384 acceptChunk(); 1385 // the format begins with two extra leading dashes, then we should be at the boundary 1386 if(pps.buffer.length < 2) 1387 return; 1388 assert(pps.buffer[0] == '-', "no leading dash"); 1389 consume(1); 1390 assert(pps.buffer[0] == '-', "no second leading dash"); 1391 consume(1); 1392 1393 pps.whatDoWeWant = 1; 1394 goto case 1; 1395 /* fallthrough */ 1396 case 1: // looking for headers 1397 // here, we should be lined up right at the boundary, which is followed by a \r\n 1398 1399 // want to keep the buffer under control in case we're under attack 1400 //stderr.writeln("here once"); 1401 //if(pps.buffer.length + chunk.length > 70 * 1024) // they should be < 1 kb really.... 1402 // throw new Exception("wtf is up with the huge mime part headers"); 1403 1404 acceptChunk(); 1405 1406 if(pps.buffer.length < pps.boundary.length) 1407 return; // not enough data, since there should always be a boundary here at least 1408 1409 if(pps.contentConsumed + pps.boundary.length + 6 == pps.expectedLength) { 1410 assert(pps.buffer.length == pps.boundary.length + 4 + 2); // --, --, and \r\n 1411 // we *should* be at the end here! 1412 assert(pps.buffer[0] == '-'); 1413 consume(1); 1414 assert(pps.buffer[0] == '-'); 1415 consume(1); 1416 1417 // the message is terminated by --BOUNDARY--\r\n (after a \r\n leading to the boundary) 1418 assert(pps.buffer[0 .. pps.boundary.length] == cast(const(ubyte[])) pps.boundary, 1419 "not lined up on boundary " ~ pps.boundary); 1420 consume(pps.boundary.length); 1421 1422 assert(pps.buffer[0] == '-'); 1423 consume(1); 1424 assert(pps.buffer[0] == '-'); 1425 consume(1); 1426 1427 assert(pps.buffer[0] == '\r'); 1428 consume(1); 1429 assert(pps.buffer[0] == '\n'); 1430 consume(1); 1431 1432 assert(pps.buffer.length == 0); 1433 assert(pps.contentConsumed == pps.expectedLength); 1434 break dataConsumptionLoop; // we're done! 1435 } else { 1436 // we're not done yet. We should be lined up on a boundary. 1437 1438 // But, we want to ensure the headers are here before we consume anything! 1439 auto headerEndLocation = locationOf(pps.buffer, "\r\n\r\n"); 1440 if(headerEndLocation == -1) 1441 return; // they *should* all be here, so we can handle them all at once. 1442 1443 assert(pps.buffer[0 .. pps.boundary.length] == cast(const(ubyte[])) pps.boundary, 1444 "not lined up on boundary " ~ pps.boundary); 1445 1446 consume(pps.boundary.length); 1447 // the boundary is always followed by a \r\n 1448 assert(pps.buffer[0] == '\r'); 1449 consume(1); 1450 assert(pps.buffer[0] == '\n'); 1451 consume(1); 1452 } 1453 1454 // re-running since by consuming the boundary, we invalidate the old index. 1455 auto headerEndLocation = locationOf(pps.buffer, "\r\n\r\n"); 1456 assert(headerEndLocation >= 0, "no header"); 1457 auto thisOnesHeaders = pps.buffer[0..headerEndLocation]; 1458 1459 consume(headerEndLocation + 4); // The +4 is the \r\n\r\n that caps it off 1460 1461 pps.thisOnesHeaders = split(cast(string) thisOnesHeaders, "\r\n"); 1462 1463 // now we'll parse the headers 1464 foreach(h; pps.thisOnesHeaders) { 1465 auto p = h.indexOf(":"); 1466 assert(p != -1, "no colon in header, got " ~ to!string(pps.thisOnesHeaders)); 1467 string hn = h[0..p]; 1468 string hv = h[p+2..$]; 1469 1470 switch(hn.toLower) { 1471 default: assert(0); 1472 case "content-disposition": 1473 auto info = hv.split("; "); 1474 foreach(i; info[1..$]) { // skipping the form-data 1475 auto o = i.split("="); // FIXME 1476 string pn = o[0]; 1477 string pv = o[1][1..$-1]; 1478 1479 if(pn == "name") { 1480 pps.piece.name = pv; 1481 } else if (pn == "filename") { 1482 pps.piece.filename = pv; 1483 pps.isFile = true; 1484 } 1485 } 1486 break; 1487 case "content-type": 1488 pps.piece.contentType = hv; 1489 break; 1490 } 1491 } 1492 1493 pps.whatDoWeWant++; // move to the next step - the data 1494 break; 1495 case 2: 1496 // when we get here, pps.buffer should contain our first chunk of data 1497 1498 if(pps.buffer.length + chunk.length > 8 * 1024 * 1024) // we might buffer quite a bit but not much 1499 throw new Exception("wtf is up with the huge mime part buffer"); 1500 1501 acceptChunk(); 1502 1503 // so the trick is, we want to process all the data up to the boundary, 1504 // but what if the chunk's end cuts the boundary off? If we're unsure, we 1505 // want to wait for the next chunk. We start by looking for the whole boundary 1506 // in the buffer somewhere. 1507 1508 auto boundaryLocation = locationOf(pps.buffer, pps.localBoundary); 1509 // assert(boundaryLocation != -1, "should have seen "~to!string(cast(ubyte[]) pps.localBoundary)~" in " ~ to!string(pps.buffer)); 1510 if(boundaryLocation != -1) { 1511 // this is easy - we can see it in it's entirety! 1512 1513 pps.piece.content ~= consume(boundaryLocation); 1514 1515 assert(pps.buffer[0] == '\r'); 1516 consume(1); 1517 assert(pps.buffer[0] == '\n'); 1518 consume(1); 1519 assert(pps.buffer[0] == '-'); 1520 consume(1); 1521 assert(pps.buffer[0] == '-'); 1522 consume(1); 1523 // the boundary here is always preceded by \r\n--, which is why we used localBoundary instead of boundary to locate it. Cut that off. 1524 pps.weHaveAPart = true; 1525 pps.whatDoWeWant = 1; // back to getting headers for the next part 1526 1527 commitPart(); // we're done here 1528 } else { 1529 // we can't see the whole thing, but what if there's a partial boundary? 1530 1531 enforce(pps.localBoundary.length < 128); // the boundary ought to be less than a line... 1532 assert(pps.localBoundary.length > 1); // should already be sane but just in case 1533 bool potentialBoundaryFound = false; 1534 1535 boundaryCheck: for(int a = 1; a < pps.localBoundary.length; a++) { 1536 // we grow the boundary a bit each time. If we think it looks the 1537 // same, better pull another chunk to be sure it's not the end. 1538 // Starting small because exiting the loop early is desirable, since 1539 // we're not keeping any ambiguity and 1 / 256 chance of exiting is 1540 // the best we can do. 1541 if(a > pps.buffer.length) 1542 break; // FIXME: is this right? 1543 assert(a <= pps.buffer.length); 1544 assert(a > 0); 1545 if(std.algorithm.endsWith(pps.buffer, pps.localBoundary[0 .. a])) { 1546 // ok, there *might* be a boundary here, so let's 1547 // not treat the end as data yet. The rest is good to 1548 // use though, since if there was a boundary there, we'd 1549 // have handled it up above after locationOf. 1550 1551 pps.piece.content ~= pps.buffer[0 .. $ - a]; 1552 consume(pps.buffer.length - a); 1553 pieceHasNewContent(); 1554 potentialBoundaryFound = true; 1555 break boundaryCheck; 1556 } 1557 } 1558 1559 if(!potentialBoundaryFound) { 1560 // we can consume the whole thing 1561 pps.piece.content ~= pps.buffer; 1562 pieceHasNewContent(); 1563 consume(pps.buffer.length); 1564 } else { 1565 // we found a possible boundary, but there was 1566 // insufficient data to be sure. 1567 assert(pps.buffer == cast(const(ubyte[])) pps.localBoundary[0 .. pps.buffer.length]); 1568 1569 return; // wait for the next chunk. 1570 } 1571 } 1572 } 1573 } while(pps.buffer.length); 1574 1575 // btw all boundaries except the first should have a \r\n before them 1576 } else { 1577 // application/x-www-form-urlencoded and application/json 1578 1579 // not using maxContentLength because that might be cranked up to allow 1580 // large file uploads. We can handle them, but a huge post[] isn't any good. 1581 if(pps.buffer.length + chunk.length > 8 * 1024 * 1024) // surely this is plenty big enough 1582 throw new Exception("wtf is up with such a gigantic form submission????"); 1583 1584 pps.buffer ~= chunk; 1585 1586 // simple handling, but it works... until someone bombs us with gigabytes of crap at least... 1587 if(pps.buffer.length == pps.expectedLength) { 1588 if(pps.needsSavedBody) 1589 pps.postBody = cast(string) pps.buffer; 1590 else 1591 pps._post = decodeVariables(cast(string) pps.buffer, "&", &allPostNamesInOrder, &allPostValuesInOrder); 1592 version(preserveData) 1593 originalPostData = pps.buffer; 1594 } else { 1595 // just for debugging 1596 } 1597 } 1598 } 1599 1600 protected void cleanUpPostDataState() { 1601 pps = PostParserState.init; 1602 } 1603 1604 /// you can override this function to somehow react 1605 /// to an upload in progress. 1606 /// 1607 /// Take note that parts of the CGI object is not yet 1608 /// initialized! Stuff from HTTP headers, including get[], is usable. 1609 /// But, none of post[] is usable, and you cannot write here. That's 1610 /// why this method is const - mutating the object won't do much anyway. 1611 /// 1612 /// My idea here was so you can output a progress bar or 1613 /// something to a cooperative client (see arsd.rtud for a potential helper) 1614 /// 1615 /// The default is to do nothing. Subclass cgi and use the 1616 /// CustomCgiMain mixin to do something here. 1617 void onRequestBodyDataReceived(size_t receivedSoFar, size_t totalExpected) const { 1618 // This space intentionally left blank. 1619 } 1620 1621 /// Initializes the cgi from completely raw HTTP data. The ir must have a Socket source. 1622 /// *closeConnection will be set to true if you should close the connection after handling this request 1623 this(BufferedInputRange ir, bool* closeConnection) { 1624 isCalledWithCommandLineArguments = false; 1625 import al = std.algorithm; 1626 1627 immutable(ubyte)[] data; 1628 1629 void rdo(const(ubyte)[] d) { 1630 //import std.stdio; writeln(d); 1631 sendAll(ir.source, d); 1632 } 1633 1634 auto ira = ir.source.remoteAddress(); 1635 auto irLocalAddress = ir.source.localAddress(); 1636 1637 ushort port = 80; 1638 if(auto ia = cast(InternetAddress) irLocalAddress) { 1639 port = ia.port; 1640 } else if(auto ia = cast(Internet6Address) irLocalAddress) { 1641 port = ia.port; 1642 } 1643 1644 // that check for UnixAddress is to work around a Phobos bug 1645 // see: https://github.com/dlang/phobos/pull/7383 1646 // but this might be more useful anyway tbh for this case 1647 version(Posix) 1648 this(ir, ira is null ? null : cast(UnixAddress) ira ? "unix:" : ira.toString(), port, 0, false, &rdo, null, closeConnection); 1649 else 1650 this(ir, ira is null ? null : ira.toString(), port, 0, false, &rdo, null, closeConnection); 1651 } 1652 1653 /** 1654 Initializes it from raw HTTP request data. GenericMain uses this when you compile with -version=embedded_httpd. 1655 1656 NOTE: If you are behind a reverse proxy, the values here might not be what you expect.... it will use X-Forwarded-For for remote IP and X-Forwarded-Host for host 1657 1658 Params: 1659 inputData = the incoming data, including headers and other raw http data. 1660 When the constructor exits, it will leave this range exactly at the start of 1661 the next request on the connection (if there is one). 1662 1663 address = the IP address of the remote user 1664 _port = the port number of the connection 1665 pathInfoStarts = the offset into the path component of the http header where the SCRIPT_NAME ends and the PATH_INFO begins. 1666 _https = if this connection is encrypted (note that the input data must not actually be encrypted) 1667 _rawDataOutput = delegate to accept response data. It should write to the socket or whatever; Cgi does all the needed processing to speak http. 1668 _flush = if _rawDataOutput buffers, this delegate should flush the buffer down the wire 1669 closeConnection = if the request asks to close the connection, *closeConnection == true. 1670 */ 1671 this( 1672 BufferedInputRange inputData, 1673 // string[] headers, immutable(ubyte)[] data, 1674 string address, ushort _port, 1675 int pathInfoStarts = 0, // use this if you know the script name, like if this is in a folder in a bigger web environment 1676 bool _https = false, 1677 void delegate(const(ubyte)[]) _rawDataOutput = null, 1678 void delegate() _flush = null, 1679 // this pointer tells if the connection is supposed to be closed after we handle this 1680 bool* closeConnection = null) 1681 { 1682 // these are all set locally so the loop works 1683 // without triggering errors in dmd 2.064 1684 // we go ahead and set them at the end of it to the this version 1685 int port; 1686 string referrer; 1687 string remoteAddress; 1688 string userAgent; 1689 string authorization; 1690 string origin; 1691 string accept; 1692 string lastEventId; 1693 bool https; 1694 string host; 1695 RequestMethod requestMethod; 1696 string requestUri; 1697 string pathInfo; 1698 string queryString; 1699 string scriptName; 1700 string[string] get; 1701 string[][string] getArray; 1702 bool keepAliveRequested; 1703 bool acceptsGzip; 1704 string cookie; 1705 1706 1707 1708 environmentVariables = cast(const) environment.toAA; 1709 1710 idlol = inputData; 1711 1712 isCalledWithCommandLineArguments = false; 1713 1714 https = _https; 1715 port = _port; 1716 1717 rawDataOutput = _rawDataOutput; 1718 flushDelegate = _flush; 1719 nph = true; 1720 1721 remoteAddress = address; 1722 1723 // streaming parser 1724 import al = std.algorithm; 1725 1726 // FIXME: tis cast is technically wrong, but Phobos deprecated al.indexOf... for some reason. 1727 auto idx = indexOf(cast(string) inputData.front(), "\r\n\r\n"); 1728 while(idx == -1) { 1729 inputData.popFront(0); 1730 idx = indexOf(cast(string) inputData.front(), "\r\n\r\n"); 1731 } 1732 1733 assert(idx != -1); 1734 1735 1736 string contentType = ""; 1737 string[string] requestHeadersHere; 1738 1739 size_t contentLength; 1740 1741 bool isChunked; 1742 1743 { 1744 import core.runtime; 1745 scriptFileName = Runtime.args[0]; 1746 } 1747 1748 1749 int headerNumber = 0; 1750 foreach(line; al.splitter(inputData.front()[0 .. idx], "\r\n")) 1751 if(line.length) { 1752 headerNumber++; 1753 auto header = cast(string) line.idup; 1754 if(headerNumber == 1) { 1755 // request line 1756 auto parts = al.splitter(header, " "); 1757 requestMethod = to!RequestMethod(parts.front); 1758 parts.popFront(); 1759 requestUri = parts.front; 1760 1761 // FIXME: the requestUri could be an absolute path!!! should I rename it or something? 1762 scriptName = requestUri[0 .. pathInfoStarts]; 1763 1764 auto question = requestUri.indexOf("?"); 1765 if(question == -1) { 1766 queryString = ""; 1767 // FIXME: double check, this might be wrong since it could be url encoded 1768 pathInfo = requestUri[pathInfoStarts..$]; 1769 } else { 1770 queryString = requestUri[question+1..$]; 1771 pathInfo = requestUri[pathInfoStarts..question]; 1772 } 1773 1774 auto ugh = decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 1775 getArray = cast(string[][string]) assumeUnique(ugh); 1776 1777 if(header.indexOf("HTTP/1.0") != -1) { 1778 http10 = true; 1779 autoBuffer = true; 1780 if(closeConnection) { 1781 // on http 1.0, close is assumed (unlike http/1.1 where we assume keep alive) 1782 *closeConnection = true; 1783 } 1784 } 1785 } else { 1786 // other header 1787 auto colon = header.indexOf(":"); 1788 if(colon == -1) 1789 throw new Exception("HTTP headers should have a colon!"); 1790 string name = header[0..colon].toLower; 1791 string value = header[colon+2..$]; // skip the colon and the space 1792 1793 requestHeadersHere[name] = value; 1794 1795 if (name == "accept") { 1796 accept = value; 1797 } 1798 else if (name == "origin") { 1799 origin = value; 1800 } 1801 else if (name == "connection") { 1802 if(value == "close" && closeConnection) 1803 *closeConnection = true; 1804 if(value.asLowerCase().canFind("keep-alive")) { 1805 keepAliveRequested = true; 1806 1807 // on http 1.0, the connection is closed by default, 1808 // but not if they request keep-alive. then we don't close 1809 // anymore - undoing the set above 1810 if(http10 && closeConnection) { 1811 *closeConnection = false; 1812 } 1813 } 1814 } 1815 else if (name == "transfer-encoding") { 1816 if(value == "chunked") 1817 isChunked = true; 1818 } 1819 else if (name == "last-event-id") { 1820 lastEventId = value; 1821 } 1822 else if (name == "authorization") { 1823 authorization = value; 1824 } 1825 else if (name == "content-type") { 1826 contentType = value; 1827 } 1828 else if (name == "content-length") { 1829 contentLength = to!size_t(value); 1830 } 1831 else if (name == "x-forwarded-for") { 1832 remoteAddress = value; 1833 } 1834 else if (name == "x-forwarded-host" || name == "host") { 1835 if(name != "host" || host is null) 1836 host = value; 1837 } 1838 // FIXME: https://tools.ietf.org/html/rfc7239 1839 else if (name == "accept-encoding") { 1840 if(value.indexOf("gzip") != -1) 1841 acceptsGzip = true; 1842 } 1843 else if (name == "user-agent") { 1844 userAgent = value; 1845 } 1846 else if (name == "referer") { 1847 referrer = value; 1848 } 1849 else if (name == "cookie") { 1850 cookie ~= value; 1851 } else if(name == "expect") { 1852 if(value == "100-continue") { 1853 // FIXME we should probably give user code a chance 1854 // to process and reject but that needs to be virtual, 1855 // perhaps part of the CGI redesign. 1856 1857 // FIXME: if size is > max content length it should 1858 // also fail at this point. 1859 _rawDataOutput(cast(ubyte[]) "HTTP/1.1 100 Continue\r\n\r\n"); 1860 } 1861 } 1862 // else 1863 // ignore it 1864 1865 } 1866 } 1867 1868 inputData.consume(idx + 4); 1869 // done 1870 1871 requestHeaders = assumeUnique(requestHeadersHere); 1872 1873 ByChunkRange dataByChunk; 1874 1875 // reading Content-Length type data 1876 // We need to read up the data we have, and write it out as a chunk. 1877 if(!isChunked) { 1878 dataByChunk = byChunk(inputData, contentLength); 1879 } else { 1880 // chunked requests happen, but not every day. Since we need to know 1881 // the content length (for now, maybe that should change), we'll buffer 1882 // the whole thing here instead of parse streaming. (I think this is what Apache does anyway in cgi modes) 1883 auto data = dechunk(inputData); 1884 1885 // set the range here 1886 dataByChunk = byChunk(data); 1887 contentLength = data.length; 1888 } 1889 1890 assert(dataByChunk !is null); 1891 1892 if(contentLength) { 1893 prepareForIncomingDataChunks(contentType, contentLength); 1894 foreach(dataChunk; dataByChunk) { 1895 handleIncomingDataChunk(dataChunk); 1896 } 1897 postArray = assumeUnique(pps._post); 1898 filesArray = assumeUnique(pps._files); 1899 files = keepLastOf(filesArray); 1900 post = keepLastOf(postArray); 1901 postBody = pps.postBody; 1902 cleanUpPostDataState(); 1903 } 1904 1905 this.port = port; 1906 this.referrer = referrer; 1907 this.remoteAddress = remoteAddress; 1908 this.userAgent = userAgent; 1909 this.authorization = authorization; 1910 this.origin = origin; 1911 this.accept = accept; 1912 this.lastEventId = lastEventId; 1913 this.https = https; 1914 this.host = host; 1915 this.requestMethod = requestMethod; 1916 this.requestUri = requestUri; 1917 this.pathInfo = pathInfo; 1918 this.queryString = queryString; 1919 1920 this.scriptName = scriptName; 1921 this.get = keepLastOf(getArray); 1922 this.getArray = cast(immutable) getArray; 1923 this.keepAliveRequested = keepAliveRequested; 1924 this.acceptsGzip = acceptsGzip; 1925 this.cookie = cookie; 1926 1927 cookiesArray = getCookieArray(); 1928 cookies = keepLastOf(cookiesArray); 1929 1930 } 1931 BufferedInputRange idlol; 1932 1933 private immutable(string[string]) keepLastOf(in string[][string] arr) { 1934 string[string] ca; 1935 foreach(k, v; arr) 1936 ca[k] = v[$-1]; 1937 1938 return assumeUnique(ca); 1939 } 1940 1941 // FIXME duplication 1942 private immutable(UploadedFile[string]) keepLastOf(in UploadedFile[][string] arr) { 1943 UploadedFile[string] ca; 1944 foreach(k, v; arr) 1945 ca[k] = v[$-1]; 1946 1947 return assumeUnique(ca); 1948 } 1949 1950 1951 private immutable(string[][string]) getCookieArray() { 1952 auto forTheLoveOfGod = decodeVariables(cookie, "; "); 1953 return assumeUnique(forTheLoveOfGod); 1954 } 1955 1956 /// Very simple method to require a basic auth username and password. 1957 /// If the http request doesn't include the required credentials, it throws a 1958 /// HTTP 401 error, and an exception. 1959 /// 1960 /// Note: basic auth does not provide great security, especially over unencrypted HTTP; 1961 /// the user's credentials are sent in plain text on every request. 1962 /// 1963 /// If you are using Apache, the HTTP_AUTHORIZATION variable may not be sent to the 1964 /// application. Either use Apache's built in methods for basic authentication, or add 1965 /// something along these lines to your server configuration: 1966 /// 1967 /// RewriteEngine On 1968 /// RewriteCond %{HTTP:Authorization} ^(.*) 1969 /// RewriteRule ^(.*) - [E=HTTP_AUTHORIZATION:%1] 1970 /// 1971 /// To ensure the necessary data is available to cgi.d. 1972 void requireBasicAuth(string user, string pass, string message = null) { 1973 if(authorization != "Basic " ~ Base64.encode(cast(immutable(ubyte)[]) (user ~ ":" ~ pass))) { 1974 setResponseStatus("401 Authorization Required"); 1975 header ("WWW-Authenticate: Basic realm=\""~message~"\""); 1976 close(); 1977 throw new Exception("Not authorized; got " ~ authorization); 1978 } 1979 } 1980 1981 /// Very simple caching controls - setCache(false) means it will never be cached. Good for rapidly updated or sensitive sites. 1982 /// setCache(true) means it will always be cached for as long as possible. Best for static content. 1983 /// Use setResponseExpires and updateResponseExpires for more control 1984 void setCache(bool allowCaching) { 1985 noCache = !allowCaching; 1986 } 1987 1988 /// Set to true and use cgi.write(data, true); to send a gzipped response to browsers 1989 /// who can accept it 1990 bool gzipResponse; 1991 1992 immutable bool acceptsGzip; 1993 immutable bool keepAliveRequested; 1994 1995 /// Set to true if and only if this was initialized with command line arguments 1996 immutable bool isCalledWithCommandLineArguments; 1997 1998 /// This gets a full url for the current request, including port, protocol, host, path, and query 1999 string getCurrentCompleteUri() const { 2000 ushort defaultPort = https ? 443 : 80; 2001 2002 string uri = "http"; 2003 if(https) 2004 uri ~= "s"; 2005 uri ~= "://"; 2006 uri ~= host; 2007 if(!(!port || port == defaultPort)) { 2008 uri ~= ":"; 2009 uri ~= to!string(port); 2010 } 2011 uri ~= requestUri; 2012 return uri; 2013 } 2014 2015 /// You can override this if your site base url isn't the same as the script name 2016 string logicalScriptName() const { 2017 return scriptName; 2018 } 2019 2020 /++ 2021 Sets the HTTP status of the response. For example, "404 File Not Found" or "500 Internal Server Error". 2022 It assumes "200 OK", and automatically changes to "302 Found" if you call setResponseLocation(). 2023 Note setResponseStatus() must be called *before* you write() any data to the output. 2024 2025 History: 2026 The `int` overload was added on January 11, 2021. 2027 +/ 2028 void setResponseStatus(string status) { 2029 assert(!outputtedResponseData); 2030 responseStatus = status; 2031 } 2032 /// ditto 2033 void setResponseStatus(int statusCode) { 2034 setResponseStatus(getHttpCodeText(statusCode)); 2035 } 2036 private string responseStatus = null; 2037 2038 /// Returns true if it is still possible to output headers 2039 bool canOutputHeaders() { 2040 return !isClosed && !outputtedResponseData; 2041 } 2042 2043 /// Sets the location header, which the browser will redirect the user to automatically. 2044 /// Note setResponseLocation() must be called *before* you write() any data to the output. 2045 /// The optional important argument is used if it's a default suggestion rather than something to insist upon. 2046 void setResponseLocation(string uri, bool important = true, string status = null) { 2047 if(!important && isCurrentResponseLocationImportant) 2048 return; // important redirects always override unimportant ones 2049 2050 if(uri is null) { 2051 responseStatus = "200 OK"; 2052 responseLocation = null; 2053 isCurrentResponseLocationImportant = important; 2054 return; // this just cancels the redirect 2055 } 2056 2057 assert(!outputtedResponseData); 2058 if(status is null) 2059 responseStatus = "302 Found"; 2060 else 2061 responseStatus = status; 2062 2063 responseLocation = uri.strip; 2064 isCurrentResponseLocationImportant = important; 2065 } 2066 protected string responseLocation = null; 2067 private bool isCurrentResponseLocationImportant = false; 2068 2069 /// Sets the Expires: http header. See also: updateResponseExpires, setPublicCaching 2070 /// The parameter is in unix_timestamp * 1000. Try setResponseExpires(getUTCtime() + SOME AMOUNT) for normal use. 2071 /// Note: the when parameter is different than setCookie's expire parameter. 2072 void setResponseExpires(long when, bool isPublic = false) { 2073 responseExpires = when; 2074 setCache(true); // need to enable caching so the date has meaning 2075 2076 responseIsPublic = isPublic; 2077 responseExpiresRelative = false; 2078 } 2079 2080 /// Sets a cache-control max-age header for whenFromNow, in seconds. 2081 void setResponseExpiresRelative(int whenFromNow, bool isPublic = false) { 2082 responseExpires = whenFromNow; 2083 setCache(true); // need to enable caching so the date has meaning 2084 2085 responseIsPublic = isPublic; 2086 responseExpiresRelative = true; 2087 } 2088 private long responseExpires = long.min; 2089 private bool responseIsPublic = false; 2090 private bool responseExpiresRelative = false; 2091 2092 /// This is like setResponseExpires, but it can be called multiple times. The setting most in the past is the one kept. 2093 /// If you have multiple functions, they all might call updateResponseExpires about their own return value. The program 2094 /// output as a whole is as cacheable as the least cachable part in the chain. 2095 2096 /// setCache(false) always overrides this - it is, by definition, the strictest anti-cache statement available. If your site outputs sensitive user data, you should probably call setCache(false) when you do, to ensure no other functions will cache the content, as it may be a privacy risk. 2097 /// Conversely, setting here overrides setCache(true), since any expiration date is in the past of infinity. 2098 void updateResponseExpires(long when, bool isPublic) { 2099 if(responseExpires == long.min) 2100 setResponseExpires(when, isPublic); 2101 else if(when < responseExpires) 2102 setResponseExpires(when, responseIsPublic && isPublic); // if any part of it is private, it all is 2103 } 2104 2105 /* 2106 /// Set to true if you want the result to be cached publically - that is, is the content shared? 2107 /// Should generally be false if the user is logged in. It assumes private cache only. 2108 /// setCache(true) also turns on public caching, and setCache(false) sets to private. 2109 void setPublicCaching(bool allowPublicCaches) { 2110 publicCaching = allowPublicCaches; 2111 } 2112 private bool publicCaching = false; 2113 */ 2114 2115 /++ 2116 History: 2117 Added January 11, 2021 2118 +/ 2119 enum SameSitePolicy { 2120 Lax, 2121 Strict, 2122 None 2123 } 2124 2125 /++ 2126 Sets an HTTP cookie, automatically encoding the data to the correct string. 2127 expiresIn is how many milliseconds in the future the cookie will expire. 2128 TIP: to make a cookie accessible from subdomains, set the domain to .yourdomain.com. 2129 Note setCookie() must be called *before* you write() any data to the output. 2130 2131 History: 2132 Parameter `sameSitePolicy` was added on January 11, 2021. 2133 +/ 2134 void setCookie(string name, string data, long expiresIn = 0, string path = null, string domain = null, bool httpOnly = false, bool secure = false, SameSitePolicy sameSitePolicy = SameSitePolicy.Lax) { 2135 assert(!outputtedResponseData); 2136 string cookie = std.uri.encodeComponent(name) ~ "="; 2137 cookie ~= std.uri.encodeComponent(data); 2138 if(path !is null) 2139 cookie ~= "; path=" ~ path; 2140 // FIXME: should I just be using max-age here? (also in cache below) 2141 if(expiresIn != 0) 2142 cookie ~= "; expires=" ~ printDate(cast(DateTime) Clock.currTime(UTC()) + dur!"msecs"(expiresIn)); 2143 if(domain !is null) 2144 cookie ~= "; domain=" ~ domain; 2145 if(secure == true) 2146 cookie ~= "; Secure"; 2147 if(httpOnly == true ) 2148 cookie ~= "; HttpOnly"; 2149 final switch(sameSitePolicy) { 2150 case SameSitePolicy.Lax: 2151 cookie ~= "; SameSite=Lax"; 2152 break; 2153 case SameSitePolicy.Strict: 2154 cookie ~= "; SameSite=Strict"; 2155 break; 2156 case SameSitePolicy.None: 2157 cookie ~= "; SameSite=None"; 2158 assert(secure); // cookie spec requires this now, see: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite 2159 break; 2160 } 2161 2162 if(auto idx = name in cookieIndexes) { 2163 responseCookies[*idx] = cookie; 2164 } else { 2165 cookieIndexes[name] = responseCookies.length; 2166 responseCookies ~= cookie; 2167 } 2168 } 2169 private string[] responseCookies; 2170 private size_t[string] cookieIndexes; 2171 2172 /// Clears a previously set cookie with the given name, path, and domain. 2173 void clearCookie(string name, string path = null, string domain = null) { 2174 assert(!outputtedResponseData); 2175 setCookie(name, "", 1, path, domain); 2176 } 2177 2178 /// Sets the content type of the response, for example "text/html" (the default) for HTML, or "image/png" for a PNG image 2179 void setResponseContentType(string ct) { 2180 assert(!outputtedResponseData); 2181 responseContentType = ct; 2182 } 2183 private string responseContentType = null; 2184 2185 /// Adds a custom header. It should be the name: value, but without any line terminator. 2186 /// For example: header("X-My-Header: Some value"); 2187 /// Note you should use the specialized functions in this object if possible to avoid 2188 /// duplicates in the output. 2189 void header(string h) { 2190 customHeaders ~= h; 2191 } 2192 2193 private string[] customHeaders; 2194 private bool websocketMode; 2195 2196 void flushHeaders(const(void)[] t, bool isAll = false) { 2197 StackBuffer buffer = StackBuffer(0); 2198 2199 prepHeaders(t, isAll, &buffer); 2200 2201 if(rawDataOutput !is null) 2202 rawDataOutput(cast(const(ubyte)[]) buffer.get()); 2203 else { 2204 stdout.rawWrite(buffer.get()); 2205 } 2206 } 2207 2208 private void prepHeaders(const(void)[] t, bool isAll, StackBuffer* buffer) { 2209 string terminator = "\n"; 2210 if(rawDataOutput !is null) 2211 terminator = "\r\n"; 2212 2213 if(responseStatus !is null) { 2214 if(nph) { 2215 if(http10) 2216 buffer.add("HTTP/1.0 ", responseStatus, terminator); 2217 else 2218 buffer.add("HTTP/1.1 ", responseStatus, terminator); 2219 } else 2220 buffer.add("Status: ", responseStatus, terminator); 2221 } else if (nph) { 2222 if(http10) 2223 buffer.add("HTTP/1.0 200 OK", terminator); 2224 else 2225 buffer.add("HTTP/1.1 200 OK", terminator); 2226 } 2227 2228 if(websocketMode) 2229 goto websocket; 2230 2231 if(nph) { // we're responsible for setting the date too according to http 1.1 2232 char[29] db = void; 2233 printDateToBuffer(cast(DateTime) Clock.currTime(UTC()), db[]); 2234 buffer.add("Date: ", db[], terminator); 2235 } 2236 2237 // FIXME: what if the user wants to set his own content-length? 2238 // The custom header function can do it, so maybe that's best. 2239 // Or we could reuse the isAll param. 2240 if(responseLocation !is null) { 2241 buffer.add("Location: ", responseLocation, terminator); 2242 } 2243 if(!noCache && responseExpires != long.min) { // an explicit expiration date is set 2244 if(responseExpiresRelative) { 2245 buffer.add("Cache-Control: ", responseIsPublic ? "public" : "private", ", max-age="); 2246 buffer.add(responseExpires); 2247 buffer.add(", no-cache=\"set-cookie, set-cookie2\"", terminator); 2248 } else { 2249 auto expires = SysTime(unixTimeToStdTime(cast(int)(responseExpires / 1000)), UTC()); 2250 char[29] db = void; 2251 printDateToBuffer(cast(DateTime) expires, db[]); 2252 buffer.add("Expires: ", db[], terminator); 2253 // FIXME: assuming everything is private unless you use nocache - generally right for dynamic pages, but not necessarily 2254 buffer.add("Cache-Control: ", (responseIsPublic ? "public" : "private"), ", no-cache=\"set-cookie, set-cookie2\""); 2255 buffer.add(terminator); 2256 } 2257 } 2258 if(responseCookies !is null && responseCookies.length > 0) { 2259 foreach(c; responseCookies) 2260 buffer.add("Set-Cookie: ", c, terminator); 2261 } 2262 if(noCache) { // we specifically do not want caching (this is actually the default) 2263 buffer.add("Cache-Control: private, no-cache=\"set-cookie\"", terminator); 2264 buffer.add("Expires: 0", terminator); 2265 buffer.add("Pragma: no-cache", terminator); 2266 } else { 2267 if(responseExpires == long.min) { // caching was enabled, but without a date set - that means assume cache forever 2268 buffer.add("Cache-Control: public", terminator); 2269 buffer.add("Expires: Tue, 31 Dec 2030 14:00:00 GMT", terminator); // FIXME: should not be more than one year in the future 2270 } 2271 } 2272 if(responseContentType !is null) { 2273 buffer.add("Content-Type: ", responseContentType, terminator); 2274 } else 2275 buffer.add("Content-Type: text/html; charset=utf-8", terminator); 2276 2277 if(gzipResponse && acceptsGzip && isAll) { // FIXME: isAll really shouldn't be necessary 2278 buffer.add("Content-Encoding: gzip", terminator); 2279 } 2280 2281 2282 if(!isAll) { 2283 if(nph && !http10) { 2284 buffer.add("Transfer-Encoding: chunked", terminator); 2285 responseChunked = true; 2286 } 2287 } else { 2288 buffer.add("Content-Length: "); 2289 buffer.add(t.length); 2290 buffer.add(terminator); 2291 if(nph && keepAliveRequested) { 2292 buffer.add("Connection: Keep-Alive", terminator); 2293 } 2294 } 2295 2296 websocket: 2297 2298 foreach(hd; customHeaders) 2299 buffer.add(hd, terminator); 2300 2301 // FIXME: what about duplicated headers? 2302 2303 // end of header indicator 2304 buffer.add(terminator); 2305 2306 outputtedResponseData = true; 2307 } 2308 2309 /// Writes the data to the output, flushing headers if they have not yet been sent. 2310 void write(const(void)[] t, bool isAll = false, bool maybeAutoClose = true) { 2311 assert(!closed, "Output has already been closed"); 2312 2313 StackBuffer buffer = StackBuffer(0); 2314 2315 if(gzipResponse && acceptsGzip && isAll) { // FIXME: isAll really shouldn't be necessary 2316 // actually gzip the data here 2317 2318 auto c = new Compress(HeaderFormat.gzip); // want gzip 2319 2320 auto data = c.compress(t); 2321 data ~= c.flush(); 2322 2323 // std.file.write("/tmp/last-item", data); 2324 2325 t = data; 2326 } 2327 2328 if(!outputtedResponseData && (!autoBuffer || isAll)) { 2329 prepHeaders(t, isAll, &buffer); 2330 } 2331 2332 if(requestMethod != RequestMethod.HEAD && t.length > 0) { 2333 if (autoBuffer && !isAll) { 2334 outputBuffer ~= cast(ubyte[]) t; 2335 } 2336 if(!autoBuffer || isAll) { 2337 if(rawDataOutput !is null) 2338 if(nph && responseChunked) { 2339 //rawDataOutput(makeChunk(cast(const(ubyte)[]) t)); 2340 // we're making the chunk here instead of in a function 2341 // to avoid unneeded gc pressure 2342 buffer.add(toHex(t.length)); 2343 buffer.add("\r\n"); 2344 buffer.add(cast(char[]) t, "\r\n"); 2345 } else { 2346 buffer.add(cast(char[]) t); 2347 } 2348 else 2349 buffer.add(cast(char[]) t); 2350 } 2351 } 2352 2353 if(rawDataOutput !is null) 2354 rawDataOutput(cast(const(ubyte)[]) buffer.get()); 2355 else 2356 stdout.rawWrite(buffer.get()); 2357 2358 if(maybeAutoClose && isAll) 2359 close(); // if you say it is all, that means we're definitely done 2360 // maybeAutoClose can be false though to avoid this (important if you call from inside close()! 2361 } 2362 2363 /++ 2364 Convenience method to set content type to json and write the string as the complete response. 2365 2366 History: 2367 Added January 16, 2020 2368 +/ 2369 void writeJson(string json) { 2370 this.setResponseContentType("application/json"); 2371 this.write(json, true); 2372 } 2373 2374 /// Flushes the pending buffer, leaving the connection open so you can send more. 2375 void flush() { 2376 if(rawDataOutput is null) 2377 stdout.flush(); 2378 else if(flushDelegate !is null) 2379 flushDelegate(); 2380 } 2381 2382 version(autoBuffer) 2383 bool autoBuffer = true; 2384 else 2385 bool autoBuffer = false; 2386 ubyte[] outputBuffer; 2387 2388 /// Flushes the buffers to the network, signifying that you are done. 2389 /// You should always call this explicitly when you are done outputting data. 2390 void close() { 2391 if(closed) 2392 return; // don't double close 2393 2394 if(!outputtedResponseData) 2395 write("", false, false); 2396 2397 // writing auto buffered data 2398 if(requestMethod != RequestMethod.HEAD && autoBuffer) { 2399 if(!nph) 2400 stdout.rawWrite(outputBuffer); 2401 else 2402 write(outputBuffer, true, false); // tell it this is everything 2403 } 2404 2405 // closing the last chunk... 2406 if(nph && rawDataOutput !is null && responseChunked) 2407 rawDataOutput(cast(const(ubyte)[]) "0\r\n\r\n"); 2408 2409 if(flushDelegate) 2410 flushDelegate(); 2411 2412 closed = true; 2413 } 2414 2415 // Closes without doing anything, shouldn't be used often 2416 void rawClose() { 2417 closed = true; 2418 } 2419 2420 /++ 2421 Gets a request variable as a specific type, or the default value of it isn't there 2422 or isn't convertible to the request type. 2423 2424 Checks both GET and POST variables, preferring the POST variable, if available. 2425 2426 A nice trick is using the default value to choose the type: 2427 2428 --- 2429 /* 2430 The return value will match the type of the default. 2431 Here, I gave 10 as a default, so the return value will 2432 be an int. 2433 2434 If the user-supplied value cannot be converted to the 2435 requested type, you will get the default value back. 2436 */ 2437 int a = cgi.request("number", 10); 2438 2439 if(cgi.get["number"] == "11") 2440 assert(a == 11); // conversion succeeds 2441 2442 if("number" !in cgi.get) 2443 assert(a == 10); // no value means you can't convert - give the default 2444 2445 if(cgi.get["number"] == "twelve") 2446 assert(a == 10); // conversion from string to int would fail, so we get the default 2447 --- 2448 2449 You can use an enum as an easy whitelist, too: 2450 2451 --- 2452 enum Operations { 2453 add, remove, query 2454 } 2455 2456 auto op = cgi.request("op", Operations.query); 2457 2458 if(cgi.get["op"] == "add") 2459 assert(op == Operations.add); 2460 if(cgi.get["op"] == "remove") 2461 assert(op == Operations.remove); 2462 if(cgi.get["op"] == "query") 2463 assert(op == Operations.query); 2464 2465 if(cgi.get["op"] == "random string") 2466 assert(op == Operations.query); // the value can't be converted to the enum, so we get the default 2467 --- 2468 +/ 2469 T request(T = string)(in string name, in T def = T.init) const nothrow { 2470 try { 2471 return 2472 (name in post) ? to!T(post[name]) : 2473 (name in get) ? to!T(get[name]) : 2474 def; 2475 } catch(Exception e) { return def; } 2476 } 2477 2478 /// Is the output already closed? 2479 bool isClosed() const { 2480 return closed; 2481 } 2482 2483 /++ 2484 Gets a session object associated with the `cgi` request. You can use different type throughout your application. 2485 +/ 2486 Session!Data getSessionObject(Data)() { 2487 if(testInProcess !is null) { 2488 // test mode 2489 auto obj = testInProcess.getSessionOverride(typeid(typeof(return))); 2490 if(obj !is null) 2491 return cast(typeof(return)) obj; 2492 else { 2493 auto o = new MockSession!Data(); 2494 testInProcess.setSessionOverride(typeid(typeof(return)), o); 2495 return o; 2496 } 2497 } else { 2498 // normal operation 2499 return new BasicDataServerSession!Data(this); 2500 } 2501 } 2502 2503 // if it is in test mode; triggers mock sessions. Used by CgiTester 2504 version(with_breaking_cgi_features) 2505 private CgiTester testInProcess; 2506 2507 /* Hooks for redirecting input and output */ 2508 private void delegate(const(ubyte)[]) rawDataOutput = null; 2509 private void delegate() flushDelegate = null; 2510 2511 /* This info is used when handling a more raw HTTP protocol */ 2512 private bool nph; 2513 private bool http10; 2514 private bool closed; 2515 private bool responseChunked = false; 2516 2517 version(preserveData) // note: this can eat lots of memory; don't use unless you're sure you need it. 2518 immutable(ubyte)[] originalPostData; 2519 2520 public immutable string postBody; 2521 alias postJson = postBody; // old name 2522 2523 /* Internal state flags */ 2524 private bool outputtedResponseData; 2525 private bool noCache = true; 2526 2527 const(string[string]) environmentVariables; 2528 2529 /** What follows is data gotten from the HTTP request. It is all fully immutable, 2530 partially because it logically is (your code doesn't change what the user requested...) 2531 and partially because I hate how bad programs in PHP change those superglobals to do 2532 all kinds of hard to follow ugliness. I don't want that to ever happen in D. 2533 2534 For some of these, you'll want to refer to the http or cgi specs for more details. 2535 */ 2536 immutable(string[string]) requestHeaders; /// All the raw headers in the request as name/value pairs. The name is stored as all lower case, but otherwise the same as it is in HTTP; words separated by dashes. For example, "cookie" or "accept-encoding". Many HTTP headers have specialized variables below for more convenience and static name checking; you should generally try to use them. 2537 2538 immutable(char[]) host; /// The hostname in the request. If one program serves multiple domains, you can use this to differentiate between them. 2539 immutable(char[]) origin; /// The origin header in the request, if present. Some HTML5 cross-domain apis set this and you should check it on those cross domain requests and websockets. 2540 immutable(char[]) userAgent; /// The browser's user-agent string. Can be used to identify the browser. 2541 immutable(char[]) pathInfo; /// This is any stuff sent after your program's name on the url, but before the query string. For example, suppose your program is named "app". If the user goes to site.com/app, pathInfo is empty. But, he can also go to site.com/app/some/sub/path; treating your program like a virtual folder. In this case, pathInfo == "/some/sub/path". 2542 immutable(char[]) scriptName; /// The full base path of your program, as seen by the user. If your program is located at site.com/programs/apps, scriptName == "/programs/apps". 2543 immutable(char[]) scriptFileName; /// The physical filename of your script 2544 immutable(char[]) authorization; /// The full authorization string from the header, undigested. Useful for implementing auth schemes such as OAuth 1.0. Note that some web servers do not forward this to the app without taking extra steps. See requireBasicAuth's comment for more info. 2545 immutable(char[]) accept; /// The HTTP accept header is the user agent telling what content types it is willing to accept. This is often */*; they accept everything, so it's not terribly useful. (The similar sounding Accept-Encoding header is handled automatically for chunking and gzipping. Simply set gzipResponse = true and cgi.d handles the details, zipping if the user's browser is willing to accept it.) 2546 immutable(char[]) lastEventId; /// The HTML 5 draft includes an EventSource() object that connects to the server, and remains open to take a stream of events. My arsd.rtud module can help with the server side part of that. The Last-Event-Id http header is defined in the draft to help handle loss of connection. When the browser reconnects to you, it sets this header to the last event id it saw, so you can catch it up. This member has the contents of that header. 2547 2548 immutable(RequestMethod) requestMethod; /// The HTTP request verb: GET, POST, etc. It is represented as an enum in cgi.d (which, like many enums, you can convert back to string with std.conv.to()). A HTTP GET is supposed to, according to the spec, not have side effects; a user can GET something over and over again and always have the same result. On all requests, the get[] and getArray[] members may be filled in. The post[] and postArray[] members are only filled in on POST methods. 2549 immutable(char[]) queryString; /// The unparsed content of the request query string - the stuff after the ? in your URL. See get[] and getArray[] for a parse view of it. Sometimes, the unparsed string is useful though if you want a custom format of data up there (probably not a good idea, unless it is really simple, like "?username" perhaps.) 2550 immutable(char[]) cookie; /// The unparsed content of the Cookie: header in the request. See also the cookies[string] member for a parsed view of the data. 2551 /** The Referer header from the request. (It is misspelled in the HTTP spec, and thus the actual request and cgi specs too, but I spelled the word correctly here because that's sane. The spec's misspelling is an implementation detail.) It contains the site url that referred the user to your program; the site that linked to you, or if you're serving images, the site that has you as an image. Also, if you're in an iframe, the referrer is the site that is framing you. 2552 2553 Important note: if the user copy/pastes your url, this is blank, and, just like with all other user data, their browsers can also lie to you. Don't rely on it for real security. 2554 */ 2555 immutable(char[]) referrer; 2556 immutable(char[]) requestUri; /// The full url if the current request, excluding the protocol and host. requestUri == scriptName ~ pathInfo ~ (queryString.length ? "?" ~ queryString : ""); 2557 2558 immutable(char[]) remoteAddress; /// The IP address of the user, as we see it. (Might not match the IP of the user's computer due to things like proxies and NAT.) 2559 2560 immutable bool https; /// Was the request encrypted via https? 2561 immutable int port; /// On what TCP port number did the server receive the request? 2562 2563 /** Here come the parsed request variables - the things that come close to PHP's _GET, _POST, etc. superglobals in content. */ 2564 2565 immutable(string[string]) get; /// The data from your query string in the url, only showing the last string of each name. If you want to handle multiple values with the same name, use getArray. This only works right if the query string is x-www-form-urlencoded; the default you see on the web with name=value pairs separated by the & character. 2566 immutable(string[string]) post; /// The data from the request's body, on POST requests. It parses application/x-www-form-urlencoded data (used by most web requests, including typical forms), and multipart/form-data requests (used by file uploads on web forms) into the same container, so you can always access them the same way. It makes no attempt to parse other content types. If you want to accept an XML Post body (for a web api perhaps), you'll need to handle the raw data yourself. 2567 immutable(string[string]) cookies; /// Separates out the cookie header into individual name/value pairs (which is how you set them!) 2568 2569 /** 2570 Represents user uploaded files. 2571 2572 When making a file upload form, be sure to follow the standard: set method="POST" and enctype="multipart/form-data" in your html <form> tag attributes. The key into this array is the name attribute on your input tag, just like with other post variables. See the comments on the UploadedFile struct for more information about the data inside, including important notes on max size and content location. 2573 */ 2574 immutable(UploadedFile[][string]) filesArray; 2575 immutable(UploadedFile[string]) files; 2576 2577 /// Use these if you expect multiple items submitted with the same name. btw, assert(get[name] is getArray[name][$-1); should pass. Same for post and cookies. 2578 /// the order of the arrays is the order the data arrives 2579 immutable(string[][string]) getArray; /// like get, but an array of values per name 2580 immutable(string[][string]) postArray; /// ditto for post 2581 immutable(string[][string]) cookiesArray; /// ditto for cookies 2582 2583 // convenience function for appending to a uri without extra ? 2584 // matches the name and effect of javascript's location.search property 2585 string search() const { 2586 if(queryString.length) 2587 return "?" ~ queryString; 2588 return ""; 2589 } 2590 2591 // FIXME: what about multiple files with the same name? 2592 private: 2593 //RequestMethod _requestMethod; 2594 } 2595 2596 /// use this for testing or other isolated things when you want it to be no-ops 2597 Cgi dummyCgi(Cgi.RequestMethod method = Cgi.RequestMethod.GET, string url = null, in ubyte[] data = null, void delegate(const(ubyte)[]) outputSink = null) { 2598 // we want to ignore, not use stdout 2599 if(outputSink is null) 2600 outputSink = delegate void(const(ubyte)[]) { }; 2601 2602 string[string] env; 2603 env["REQUEST_METHOD"] = to!string(method); 2604 env["CONTENT_LENGTH"] = to!string(data.length); 2605 2606 auto cgi = new Cgi( 2607 0, 2608 env, 2609 { return data; }, 2610 outputSink, 2611 null); 2612 2613 return cgi; 2614 } 2615 2616 /++ 2617 A helper test class for request handler unittests. 2618 +/ 2619 version(with_breaking_cgi_features) 2620 class CgiTester { 2621 private { 2622 SessionObject[TypeInfo] mockSessions; 2623 SessionObject getSessionOverride(TypeInfo ti) { 2624 if(auto o = ti in mockSessions) 2625 return *o; 2626 else 2627 return null; 2628 } 2629 void setSessionOverride(TypeInfo ti, SessionObject so) { 2630 mockSessions[ti] = so; 2631 } 2632 } 2633 2634 /++ 2635 Gets (and creates if necessary) a mock session object for this test. Note 2636 it will be the same one used for any test operations through this CgiTester instance. 2637 +/ 2638 Session!Data getSessionObject(Data)() { 2639 auto obj = getSessionOverride(typeid(typeof(return))); 2640 if(obj !is null) 2641 return cast(typeof(return)) obj; 2642 else { 2643 auto o = new MockSession!Data(); 2644 setSessionOverride(typeid(typeof(return)), o); 2645 return o; 2646 } 2647 } 2648 2649 /++ 2650 Pass a reference to your request handler when creating the tester. 2651 +/ 2652 this(void function(Cgi) requestHandler) { 2653 this.requestHandler = requestHandler; 2654 } 2655 2656 /++ 2657 You can check response information with these methods after you call the request handler. 2658 +/ 2659 struct Response { 2660 int code; 2661 string[string] headers; 2662 string responseText; 2663 ubyte[] responseBody; 2664 } 2665 2666 /++ 2667 Executes a test request on your request handler, and returns the response. 2668 2669 Params: 2670 url = The URL to test. Should be an absolute path, but excluding domain. e.g. `"/test"`. 2671 args = additional arguments. Same format as cgi's command line handler. 2672 +/ 2673 Response GET(string url, string[] args = null) { 2674 return executeTest("GET", url, args); 2675 } 2676 /// ditto 2677 Response POST(string url, string[] args = null) { 2678 return executeTest("POST", url, args); 2679 } 2680 2681 /// ditto 2682 Response executeTest(string method, string url, string[] args) { 2683 ubyte[] outputtedRawData; 2684 void outputSink(const(ubyte)[] data) { 2685 outputtedRawData ~= data; 2686 } 2687 auto cgi = new Cgi(["test", method, url] ~ args, &outputSink); 2688 cgi.testInProcess = this; 2689 scope(exit) cgi.dispose(); 2690 2691 requestHandler(cgi); 2692 2693 cgi.close(); 2694 2695 Response response; 2696 2697 if(outputtedRawData.length) { 2698 enum LINE = "\r\n"; 2699 2700 auto idx = outputtedRawData.locationOf(LINE ~ LINE); 2701 assert(idx != -1, to!string(outputtedRawData)); 2702 auto headers = cast(string) outputtedRawData[0 .. idx]; 2703 response.code = 200; 2704 while(headers.length) { 2705 auto i = headers.locationOf(LINE); 2706 if(i == -1) i = cast(int) headers.length; 2707 2708 auto header = headers[0 .. i]; 2709 2710 auto c = header.locationOf(":"); 2711 if(c != -1) { 2712 auto name = header[0 .. c]; 2713 auto value = header[c + 2 ..$]; 2714 2715 if(name == "Status") 2716 response.code = value[0 .. value.locationOf(" ")].to!int; 2717 2718 response.headers[name] = value; 2719 } else { 2720 assert(0); 2721 } 2722 2723 if(i != headers.length) 2724 i += 2; 2725 headers = headers[i .. $]; 2726 } 2727 response.responseBody = outputtedRawData[idx + 4 .. $]; 2728 response.responseText = cast(string) response.responseBody; 2729 } 2730 2731 return response; 2732 } 2733 2734 private void function(Cgi) requestHandler; 2735 } 2736 2737 2738 // should this be a separate module? Probably, but that's a hassle. 2739 2740 /// Makes a data:// uri that can be used as links in most newer browsers (IE8+). 2741 string makeDataUrl(string mimeType, in void[] data) { 2742 auto data64 = Base64.encode(cast(const(ubyte[])) data); 2743 return "data:" ~ mimeType ~ ";base64," ~ assumeUnique(data64); 2744 } 2745 2746 // FIXME: I don't think this class correctly decodes/encodes the individual parts 2747 /// Represents a url that can be broken down or built up through properties 2748 struct Uri { 2749 alias toString this; // blargh idk a url really is a string, but should it be implicit? 2750 2751 // scheme//userinfo@host:port/path?query#fragment 2752 2753 string scheme; /// e.g. "http" in "http://example.com/" 2754 string userinfo; /// the username (and possibly a password) in the uri 2755 string host; /// the domain name 2756 int port; /// port number, if given. Will be zero if a port was not explicitly given 2757 string path; /// e.g. "/folder/file.html" in "http://example.com/folder/file.html" 2758 string query; /// the stuff after the ? in a uri 2759 string fragment; /// the stuff after the # in a uri. 2760 2761 // idk if i want to keep these, since the functions they wrap are used many, many, many times in existing code, so this is either an unnecessary alias or a gratuitous break of compatibility 2762 // the decode ones need to keep different names anyway because we can't overload on return values... 2763 static string encode(string s) { return std.uri.encodeComponent(s); } 2764 static string encode(string[string] s) { return encodeVariables(s); } 2765 static string encode(string[][string] s) { return encodeVariables(s); } 2766 2767 /// Breaks down a uri string to its components 2768 this(string uri) { 2769 reparse(uri); 2770 } 2771 2772 private void reparse(string uri) { 2773 // from RFC 3986 2774 // the ctRegex triples the compile time and makes ugly errors for no real benefit 2775 // it was a nice experiment but just not worth it. 2776 // enum ctr = ctRegex!r"^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?"; 2777 /* 2778 Captures: 2779 0 = whole url 2780 1 = scheme, with : 2781 2 = scheme, no : 2782 3 = authority, with // 2783 4 = authority, no // 2784 5 = path 2785 6 = query string, with ? 2786 7 = query string, no ? 2787 8 = anchor, with # 2788 9 = anchor, no # 2789 */ 2790 // Yikes, even regular, non-CT regex is also unacceptably slow to compile. 1.9s on my computer! 2791 // instead, I will DIY and cut that down to 0.6s on the same computer. 2792 /* 2793 2794 Note that authority is 2795 user:password@domain:port 2796 where the user:password@ part is optional, and the :port is optional. 2797 2798 Regex translation: 2799 2800 Scheme cannot have :, /, ?, or # in it, and must have one or more chars and end in a :. It is optional, but must be first. 2801 Authority must start with //, but cannot have any other /, ?, or # in it. It is optional. 2802 Path cannot have any ? or # in it. It is optional. 2803 Query must start with ? and must not have # in it. It is optional. 2804 Anchor must start with # and can have anything else in it to end of string. It is optional. 2805 */ 2806 2807 this = Uri.init; // reset all state 2808 2809 // empty uri = nothing special 2810 if(uri.length == 0) { 2811 return; 2812 } 2813 2814 size_t idx; 2815 2816 scheme_loop: foreach(char c; uri[idx .. $]) { 2817 switch(c) { 2818 case ':': 2819 case '/': 2820 case '?': 2821 case '#': 2822 break scheme_loop; 2823 default: 2824 } 2825 idx++; 2826 } 2827 2828 if(idx == 0 && uri[idx] == ':') { 2829 // this is actually a path! we skip way ahead 2830 goto path_loop; 2831 } 2832 2833 if(idx == uri.length) { 2834 // the whole thing is a path, apparently 2835 path = uri; 2836 return; 2837 } 2838 2839 if(idx > 0 && uri[idx] == ':') { 2840 scheme = uri[0 .. idx]; 2841 idx++; 2842 } else { 2843 // we need to rewind; it found a / but no :, so the whole thing is prolly a path... 2844 idx = 0; 2845 } 2846 2847 if(idx + 2 < uri.length && uri[idx .. idx + 2] == "//") { 2848 // we have an authority.... 2849 idx += 2; 2850 2851 auto authority_start = idx; 2852 authority_loop: foreach(char c; uri[idx .. $]) { 2853 switch(c) { 2854 case '/': 2855 case '?': 2856 case '#': 2857 break authority_loop; 2858 default: 2859 } 2860 idx++; 2861 } 2862 2863 auto authority = uri[authority_start .. idx]; 2864 2865 auto idx2 = authority.indexOf("@"); 2866 if(idx2 != -1) { 2867 userinfo = authority[0 .. idx2]; 2868 authority = authority[idx2 + 1 .. $]; 2869 } 2870 2871 if(authority.length && authority[0] == '[') { 2872 // ipv6 address special casing 2873 idx2 = authority.indexOf(']'); 2874 if(idx2 != -1) { 2875 auto end = authority[idx2 + 1 .. $]; 2876 if(end.length && end[0] == ':') 2877 idx2 = idx2 + 1; 2878 else 2879 idx2 = -1; 2880 } 2881 } else { 2882 idx2 = authority.indexOf(":"); 2883 } 2884 2885 if(idx2 == -1) { 2886 port = 0; // 0 means not specified; we should use the default for the scheme 2887 host = authority; 2888 } else { 2889 host = authority[0 .. idx2]; 2890 port = to!int(authority[idx2 + 1 .. $]); 2891 } 2892 } 2893 2894 path_loop: 2895 auto path_start = idx; 2896 2897 foreach(char c; uri[idx .. $]) { 2898 if(c == '?' || c == '#') 2899 break; 2900 idx++; 2901 } 2902 2903 path = uri[path_start .. idx]; 2904 2905 if(idx == uri.length) 2906 return; // nothing more to examine... 2907 2908 if(uri[idx] == '?') { 2909 idx++; 2910 auto query_start = idx; 2911 foreach(char c; uri[idx .. $]) { 2912 if(c == '#') 2913 break; 2914 idx++; 2915 } 2916 query = uri[query_start .. idx]; 2917 } 2918 2919 if(idx < uri.length && uri[idx] == '#') { 2920 idx++; 2921 fragment = uri[idx .. $]; 2922 } 2923 2924 // uriInvalidated = false; 2925 } 2926 2927 private string rebuildUri() const { 2928 string ret; 2929 if(scheme.length) 2930 ret ~= scheme ~ ":"; 2931 if(userinfo.length || host.length) 2932 ret ~= "//"; 2933 if(userinfo.length) 2934 ret ~= userinfo ~ "@"; 2935 if(host.length) 2936 ret ~= host; 2937 if(port) 2938 ret ~= ":" ~ to!string(port); 2939 2940 ret ~= path; 2941 2942 if(query.length) 2943 ret ~= "?" ~ query; 2944 2945 if(fragment.length) 2946 ret ~= "#" ~ fragment; 2947 2948 // uri = ret; 2949 // uriInvalidated = false; 2950 return ret; 2951 } 2952 2953 /// Converts the broken down parts back into a complete string 2954 string toString() const { 2955 // if(uriInvalidated) 2956 return rebuildUri(); 2957 } 2958 2959 /// Returns a new absolute Uri given a base. It treats this one as 2960 /// relative where possible, but absolute if not. (If protocol, domain, or 2961 /// other info is not set, the new one inherits it from the base.) 2962 /// 2963 /// Browsers use a function like this to figure out links in html. 2964 Uri basedOn(in Uri baseUrl) const { 2965 Uri n = this; // copies 2966 // n.uriInvalidated = true; // make sure we regenerate... 2967 2968 // userinfo is not inherited... is this wrong? 2969 2970 // if anything is given in the existing url, we don't use the base anymore. 2971 if(n.scheme.empty) { 2972 n.scheme = baseUrl.scheme; 2973 if(n.host.empty) { 2974 n.host = baseUrl.host; 2975 if(n.port == 0) { 2976 n.port = baseUrl.port; 2977 if(n.path.length > 0 && n.path[0] != '/') { 2978 auto b = baseUrl.path[0 .. baseUrl.path.lastIndexOf("/") + 1]; 2979 if(b.length == 0) 2980 b = "/"; 2981 n.path = b ~ n.path; 2982 } else if(n.path.length == 0) { 2983 n.path = baseUrl.path; 2984 } 2985 } 2986 } 2987 } 2988 2989 n.removeDots(); 2990 2991 return n; 2992 } 2993 2994 void removeDots() { 2995 auto parts = this.path.split("/"); 2996 string[] toKeep; 2997 foreach(part; parts) { 2998 if(part == ".") { 2999 continue; 3000 } else if(part == "..") { 3001 //if(toKeep.length > 1) 3002 toKeep = toKeep[0 .. $-1]; 3003 //else 3004 //toKeep = [""]; 3005 continue; 3006 } else { 3007 //if(toKeep.length && toKeep[$-1].length == 0 && part.length == 0) 3008 //continue; // skip a `//` situation 3009 toKeep ~= part; 3010 } 3011 } 3012 3013 auto path = toKeep.join("/"); 3014 if(path.length && path[0] != '/') 3015 path = "/" ~ path; 3016 3017 this.path = path; 3018 } 3019 3020 unittest { 3021 auto uri = Uri("test.html"); 3022 assert(uri.path == "test.html"); 3023 uri = Uri("path/1/lol"); 3024 assert(uri.path == "path/1/lol"); 3025 uri = Uri("http://me@example.com"); 3026 assert(uri.scheme == "http"); 3027 assert(uri.userinfo == "me"); 3028 assert(uri.host == "example.com"); 3029 uri = Uri("http://example.com/#a"); 3030 assert(uri.scheme == "http"); 3031 assert(uri.host == "example.com"); 3032 assert(uri.fragment == "a"); 3033 uri = Uri("#foo"); 3034 assert(uri.fragment == "foo"); 3035 uri = Uri("?lol"); 3036 assert(uri.query == "lol"); 3037 uri = Uri("#foo?lol"); 3038 assert(uri.fragment == "foo?lol"); 3039 uri = Uri("?lol#foo"); 3040 assert(uri.fragment == "foo"); 3041 assert(uri.query == "lol"); 3042 3043 uri = Uri("http://127.0.0.1/"); 3044 assert(uri.host == "127.0.0.1"); 3045 assert(uri.port == 0); 3046 3047 uri = Uri("http://127.0.0.1:123/"); 3048 assert(uri.host == "127.0.0.1"); 3049 assert(uri.port == 123); 3050 3051 uri = Uri("http://[ff:ff::0]/"); 3052 assert(uri.host == "[ff:ff::0]"); 3053 3054 uri = Uri("http://[ff:ff::0]:123/"); 3055 assert(uri.host == "[ff:ff::0]"); 3056 assert(uri.port == 123); 3057 } 3058 3059 // This can sometimes be a big pain in the butt for me, so lots of copy/paste here to cover 3060 // the possibilities. 3061 unittest { 3062 auto url = Uri("cool.html"); // checking relative links 3063 3064 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/what/cool.html"); 3065 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/what/cool.html"); 3066 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/what/cool.html"); 3067 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/cool.html"); 3068 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/cool.html"); 3069 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/what/cool.html"); 3070 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/what/cool.html"); 3071 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/what/cool.html"); 3072 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/cool.html"); 3073 3074 url = Uri("/something/cool.html"); // same server, different path 3075 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/something/cool.html"); 3076 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/something/cool.html"); 3077 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/something/cool.html"); 3078 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/something/cool.html"); 3079 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/something/cool.html"); 3080 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/something/cool.html"); 3081 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/something/cool.html"); 3082 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/something/cool.html"); 3083 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/something/cool.html"); 3084 3085 url = Uri("?query=answer"); // same path. server, protocol, and port, just different query string and fragment 3086 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/what/test.html?query=answer"); 3087 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/what/test.html?query=answer"); 3088 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/what/?query=answer"); 3089 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/?query=answer"); 3090 assert(url.basedOn(Uri("http://test.com")) == "http://test.com?query=answer"); 3091 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/what/test.html?query=answer"); 3092 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/what/test.html?query=answer"); 3093 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/what/test.html?query=answer"); 3094 assert(url.basedOn(Uri("http://test.com")) == "http://test.com?query=answer"); 3095 3096 url = Uri("/test/bar"); 3097 assert(Uri("./").basedOn(url) == "/test/", Uri("./").basedOn(url)); 3098 assert(Uri("../").basedOn(url) == "/"); 3099 3100 url = Uri("http://example.com/"); 3101 assert(Uri("../foo").basedOn(url) == "http://example.com/foo"); 3102 3103 //auto uriBefore = url; 3104 url = Uri("#anchor"); // everything should remain the same except the anchor 3105 //uriBefore.anchor = "anchor"); 3106 //assert(url == uriBefore); 3107 3108 url = Uri("//example.com"); // same protocol, but different server. the path here should be blank. 3109 3110 url = Uri("//example.com/example.html"); // same protocol, but different server and path 3111 3112 url = Uri("http://example.com/test.html"); // completely absolute link should never be modified 3113 3114 url = Uri("http://example.com"); // completely absolute link should never be modified, even if it has no path 3115 3116 // FIXME: add something for port too 3117 } 3118 3119 // these are like javascript's location.search and location.hash 3120 string search() const { 3121 return query.length ? ("?" ~ query) : ""; 3122 } 3123 string hash() const { 3124 return fragment.length ? ("#" ~ fragment) : ""; 3125 } 3126 } 3127 3128 3129 /* 3130 for session, see web.d 3131 */ 3132 3133 /// breaks down a url encoded string 3134 string[][string] decodeVariables(string data, string separator = "&", string[]* namesInOrder = null, string[]* valuesInOrder = null) { 3135 auto vars = data.split(separator); 3136 string[][string] _get; 3137 foreach(var; vars) { 3138 auto equal = var.indexOf("="); 3139 string name; 3140 string value; 3141 if(equal == -1) { 3142 name = decodeComponent(var); 3143 value = ""; 3144 } else { 3145 //_get[decodeComponent(var[0..equal])] ~= decodeComponent(var[equal + 1 .. $].replace("+", " ")); 3146 // stupid + -> space conversion. 3147 name = decodeComponent(var[0..equal].replace("+", " ")); 3148 value = decodeComponent(var[equal + 1 .. $].replace("+", " ")); 3149 } 3150 3151 _get[name] ~= value; 3152 if(namesInOrder) 3153 (*namesInOrder) ~= name; 3154 if(valuesInOrder) 3155 (*valuesInOrder) ~= value; 3156 } 3157 return _get; 3158 } 3159 3160 /// breaks down a url encoded string, but only returns the last value of any array 3161 string[string] decodeVariablesSingle(string data) { 3162 string[string] va; 3163 auto varArray = decodeVariables(data); 3164 foreach(k, v; varArray) 3165 va[k] = v[$-1]; 3166 3167 return va; 3168 } 3169 3170 /// url encodes the whole string 3171 string encodeVariables(in string[string] data) { 3172 string ret; 3173 3174 bool outputted = false; 3175 foreach(k, v; data) { 3176 if(outputted) 3177 ret ~= "&"; 3178 else 3179 outputted = true; 3180 3181 ret ~= std.uri.encodeComponent(k) ~ "=" ~ std.uri.encodeComponent(v); 3182 } 3183 3184 return ret; 3185 } 3186 3187 /// url encodes a whole string 3188 string encodeVariables(in string[][string] data) { 3189 string ret; 3190 3191 bool outputted = false; 3192 foreach(k, arr; data) { 3193 foreach(v; arr) { 3194 if(outputted) 3195 ret ~= "&"; 3196 else 3197 outputted = true; 3198 ret ~= std.uri.encodeComponent(k) ~ "=" ~ std.uri.encodeComponent(v); 3199 } 3200 } 3201 3202 return ret; 3203 } 3204 3205 /// Encodes all but the explicitly unreserved characters per rfc 3986 3206 /// Alphanumeric and -_.~ are the only ones left unencoded 3207 /// name is borrowed from php 3208 string rawurlencode(in char[] data) { 3209 string ret; 3210 ret.reserve(data.length * 2); 3211 foreach(char c; data) { 3212 if( 3213 (c >= 'a' && c <= 'z') || 3214 (c >= 'A' && c <= 'Z') || 3215 (c >= '0' && c <= '9') || 3216 c == '-' || c == '_' || c == '.' || c == '~') 3217 { 3218 ret ~= c; 3219 } else { 3220 ret ~= '%'; 3221 // since we iterate on char, this should give us the octets of the full utf8 string 3222 ret ~= toHexUpper(c); 3223 } 3224 } 3225 3226 return ret; 3227 } 3228 3229 3230 // http helper functions 3231 3232 // for chunked responses (which embedded http does whenever possible) 3233 version(none) // this is moved up above to avoid making a copy of the data 3234 const(ubyte)[] makeChunk(const(ubyte)[] data) { 3235 const(ubyte)[] ret; 3236 3237 ret = cast(const(ubyte)[]) toHex(data.length); 3238 ret ~= cast(const(ubyte)[]) "\r\n"; 3239 ret ~= data; 3240 ret ~= cast(const(ubyte)[]) "\r\n"; 3241 3242 return ret; 3243 } 3244 3245 string toHex(long num) { 3246 string ret; 3247 while(num) { 3248 int v = num % 16; 3249 num /= 16; 3250 char d = cast(char) ((v < 10) ? v + '0' : (v-10) + 'a'); 3251 ret ~= d; 3252 } 3253 3254 return to!string(array(ret.retro)); 3255 } 3256 3257 string toHexUpper(long num) { 3258 string ret; 3259 while(num) { 3260 int v = num % 16; 3261 num /= 16; 3262 char d = cast(char) ((v < 10) ? v + '0' : (v-10) + 'A'); 3263 ret ~= d; 3264 } 3265 3266 if(ret.length == 1) 3267 ret ~= "0"; // url encoding requires two digits and that's what this function is used for... 3268 3269 return to!string(array(ret.retro)); 3270 } 3271 3272 3273 // the generic mixins 3274 3275 /// Use this instead of writing your own main 3276 mixin template GenericMain(alias fun, long maxContentLength = defaultMaxContentLength) { 3277 mixin CustomCgiMain!(Cgi, fun, maxContentLength); 3278 } 3279 3280 /++ 3281 Boilerplate mixin for a main function that uses the [dispatcher] function. 3282 3283 You can send `typeof(null)` as the `Presenter` argument to use a generic one. 3284 3285 History: 3286 Added July 9, 2021 3287 +/ 3288 mixin template DispatcherMain(Presenter, DispatcherArgs...) { 3289 /++ 3290 Handler to the generated presenter you can use from your objects, etc. 3291 +/ 3292 Presenter activePresenter; 3293 3294 /++ 3295 Request handler that creates the presenter then forwards to the [dispatcher] function. 3296 Renders 404 if the dispatcher did not handle the request. 3297 3298 Will automatically serve the presenter.style and presenter.script as "style.css" and "script.js" 3299 +/ 3300 void handler(Cgi cgi) { 3301 auto presenter = new Presenter; 3302 activePresenter = presenter; 3303 scope(exit) activePresenter = null; 3304 3305 if(cgi.dispatcher!DispatcherArgs(presenter)) 3306 return; 3307 3308 switch(cgi.pathInfo) { 3309 case "/style.css": 3310 cgi.setCache(true); 3311 cgi.setResponseContentType("text/css"); 3312 cgi.write(presenter.style(), true); 3313 break; 3314 case "/script.js": 3315 cgi.setCache(true); 3316 cgi.setResponseContentType("application/javascript"); 3317 cgi.write(presenter.script(), true); 3318 break; 3319 default: 3320 presenter.renderBasicError(cgi, 404); 3321 } 3322 } 3323 mixin GenericMain!handler; 3324 } 3325 3326 mixin template DispatcherMain(DispatcherArgs...) if(!is(DispatcherArgs[0] : WebPresenter!T, T)) { 3327 class GenericPresenter : WebPresenter!GenericPresenter {} 3328 mixin DispatcherMain!(GenericPresenter, DispatcherArgs); 3329 } 3330 3331 private string simpleHtmlEncode(string s) { 3332 return s.replace("&", "&").replace("<", "<").replace(">", ">").replace("\n", "<br />\n"); 3333 } 3334 3335 string messageFromException(Throwable t) { 3336 string message; 3337 if(t !is null) { 3338 debug message = t.toString(); 3339 else message = "An unexpected error has occurred."; 3340 } else { 3341 message = "Unknown error"; 3342 } 3343 return message; 3344 } 3345 3346 string plainHttpError(bool isCgi, string type, Throwable t) { 3347 auto message = messageFromException(t); 3348 message = simpleHtmlEncode(message); 3349 3350 return format("%s %s\r\nContent-Length: %s\r\n\r\n%s", 3351 isCgi ? "Status:" : "HTTP/1.0", 3352 type, message.length, message); 3353 } 3354 3355 // returns true if we were able to recover reasonably 3356 bool handleException(Cgi cgi, Throwable t) { 3357 if(cgi.isClosed) { 3358 // if the channel has been explicitly closed, we can't handle it here 3359 return true; 3360 } 3361 3362 if(cgi.outputtedResponseData) { 3363 // the headers are sent, but the channel is open... since it closes if all was sent, we can append an error message here. 3364 return false; // but I don't want to, since I don't know what condition the output is in; I don't want to inject something (nor check the content-type for that matter. So we say it was not a clean handling. 3365 } else { 3366 // no headers are sent, we can send a full blown error and recover 3367 cgi.setCache(false); 3368 cgi.setResponseContentType("text/html"); 3369 cgi.setResponseLocation(null); // cancel the redirect 3370 cgi.setResponseStatus("500 Internal Server Error"); 3371 cgi.write(simpleHtmlEncode(messageFromException(t))); 3372 cgi.close(); 3373 return true; 3374 } 3375 } 3376 3377 bool isCgiRequestMethod(string s) { 3378 s = s.toUpper(); 3379 if(s == "COMMANDLINE") 3380 return true; 3381 foreach(member; __traits(allMembers, Cgi.RequestMethod)) 3382 if(s == member) 3383 return true; 3384 return false; 3385 } 3386 3387 /// If you want to use a subclass of Cgi with generic main, use this mixin. 3388 mixin template CustomCgiMain(CustomCgi, alias fun, long maxContentLength = defaultMaxContentLength) if(is(CustomCgi : Cgi)) { 3389 // kinda hacky - the T... is passed to Cgi's constructor in standard cgi mode, and ignored elsewhere 3390 void main(string[] args) { 3391 cgiMainImpl!(fun, CustomCgi, maxContentLength)(args); 3392 } 3393 } 3394 3395 version(embedded_httpd_processes) 3396 __gshared int processPoolSize = 8; 3397 3398 // Returns true if run. You should exit the program after that. 3399 bool tryAddonServers(string[] args) { 3400 if(args.length > 1) { 3401 // run the special separate processes if needed 3402 switch(args[1]) { 3403 case "--websocket-server": 3404 version(with_addon_servers) 3405 websocketServers[args[2]](args[3 .. $]); 3406 else 3407 printf("Add-on servers not compiled in.\n"); 3408 return true; 3409 case "--websocket-servers": 3410 import core.demangle; 3411 version(with_addon_servers_connections) 3412 foreach(k, v; websocketServers) 3413 writeln(k, "\t", demangle(k)); 3414 return true; 3415 case "--session-server": 3416 version(with_addon_servers) 3417 runSessionServer(); 3418 else 3419 printf("Add-on servers not compiled in.\n"); 3420 return true; 3421 case "--event-server": 3422 version(with_addon_servers) 3423 runEventServer(); 3424 else 3425 printf("Add-on servers not compiled in.\n"); 3426 return true; 3427 case "--timer-server": 3428 version(with_addon_servers) 3429 runTimerServer(); 3430 else 3431 printf("Add-on servers not compiled in.\n"); 3432 return true; 3433 case "--timed-jobs": 3434 import core.demangle; 3435 version(with_addon_servers_connections) 3436 foreach(k, v; scheduledJobHandlers) 3437 writeln(k, "\t", demangle(k)); 3438 return true; 3439 case "--timed-job": 3440 scheduledJobHandlers[args[2]](args[3 .. $]); 3441 return true; 3442 default: 3443 // intentionally blank - do nothing and carry on to run normally 3444 } 3445 } 3446 return false; 3447 } 3448 3449 /// Tries to simulate a request from the command line. Returns true if it does, false if it didn't find the args. 3450 bool trySimulatedRequest(alias fun, CustomCgi = Cgi)(string[] args) if(is(CustomCgi : Cgi)) { 3451 // we support command line thing for easy testing everywhere 3452 // it needs to be called ./app method uri [other args...] 3453 if(args.length >= 3 && isCgiRequestMethod(args[1])) { 3454 Cgi cgi = new CustomCgi(args); 3455 scope(exit) cgi.dispose(); 3456 fun(cgi); 3457 cgi.close(); 3458 return true; 3459 } 3460 return false; 3461 } 3462 3463 /++ 3464 A server control and configuration struct, as a potential alternative to calling [GenericMain] or [cgiMainImpl]. See the source of [cgiMainImpl] to an example of how you can use it. 3465 3466 History: 3467 Added Sept 26, 2020 (release version 8.5). 3468 +/ 3469 struct RequestServer { 3470 /// 3471 string listeningHost = defaultListeningHost(); 3472 /// 3473 ushort listeningPort = defaultListeningPort(); 3474 3475 /// 3476 this(string defaultHost, ushort defaultPort) { 3477 this.listeningHost = defaultHost; 3478 this.listeningPort = defaultPort; 3479 } 3480 3481 /// 3482 this(ushort defaultPort) { 3483 listeningPort = defaultPort; 3484 } 3485 3486 /// Reads the args into the other values. 3487 void configureFromCommandLine(string[] args) { 3488 bool foundPort = false; 3489 bool foundHost = false; 3490 bool foundUid = false; 3491 bool foundGid = false; 3492 foreach(arg; args) { 3493 if(foundPort) { 3494 listeningPort = to!ushort(arg); 3495 foundPort = false; 3496 } 3497 if(foundHost) { 3498 listeningHost = arg; 3499 foundHost = false; 3500 } 3501 if(foundUid) { 3502 privDropUserId = to!int(arg); 3503 foundUid = false; 3504 } 3505 if(foundGid) { 3506 privDropGroupId = to!int(arg); 3507 foundGid = false; 3508 } 3509 if(arg == "--listening-host" || arg == "-h" || arg == "/listening-host") 3510 foundHost = true; 3511 else if(arg == "--port" || arg == "-p" || arg == "/port" || arg == "--listening-port") 3512 foundPort = true; 3513 else if(arg == "--uid") 3514 foundUid = true; 3515 else if(arg == "--gid") 3516 foundGid = true; 3517 } 3518 } 3519 3520 // FIXME: the privDropUserId/group id need to be set in here instead of global 3521 3522 /++ 3523 Serves a single HTTP request on this thread, with an embedded server, then stops. Designed for cases like embedded oauth responders 3524 3525 History: 3526 Added Oct 10, 2020. 3527 Example: 3528 3529 --- 3530 import arsd.cgi; 3531 void main() { 3532 RequestServer server = RequestServer("127.0.0.1", 6789); 3533 string oauthCode; 3534 string oauthScope; 3535 server.serveHttpOnce!((cgi) { 3536 oauthCode = cgi.request("code"); 3537 oauthScope = cgi.request("scope"); 3538 cgi.write("Thank you, please return to the application."); 3539 }); 3540 // use the code and scope given 3541 } 3542 --- 3543 +/ 3544 void serveHttpOnce(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3545 import std.socket; 3546 3547 bool tcp; 3548 void delegate() cleanup; 3549 auto socket = startListening(listeningHost, listeningPort, tcp, cleanup, 1); 3550 auto connection = socket.accept(); 3551 doThreadHttpConnectionGuts!(CustomCgi, fun, true)(connection); 3552 3553 if(cleanup) 3554 cleanup(); 3555 } 3556 3557 /++ 3558 Starts serving requests according to the current configuration. 3559 +/ 3560 void serve(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3561 version(netman_httpd) { 3562 // Obsolete! 3563 3564 import arsd.httpd; 3565 // what about forwarding the other constructor args? 3566 // this probably needs a whole redoing... 3567 serveHttp!CustomCgi(&fun, listeningPort);//5005); 3568 return; 3569 } else 3570 version(embedded_httpd_processes) { 3571 serveEmbeddedHttpdProcesses!(fun, CustomCgi)(this); 3572 } else 3573 version(embedded_httpd_threads) { 3574 serveEmbeddedHttp!(fun, CustomCgi, maxContentLength)(); 3575 } else 3576 version(scgi) { 3577 serveScgi!(fun, CustomCgi, maxContentLength)(); 3578 } else 3579 version(fastcgi) { 3580 serveFastCgi!(fun, CustomCgi, maxContentLength)(this); 3581 } else 3582 version(stdio_http) { 3583 serveSingleHttpConnectionOnStdio!(fun, CustomCgi, maxContentLength)(); 3584 } else { 3585 //version=plain_cgi; 3586 handleCgiRequest!(fun, CustomCgi, maxContentLength)(); 3587 } 3588 } 3589 3590 void serveEmbeddedHttp(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3591 auto manager = new ListeningConnectionManager(listeningHost, listeningPort, &doThreadHttpConnection!(CustomCgi, fun)); 3592 manager.listen(); 3593 } 3594 void serveScgi(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3595 auto manager = new ListeningConnectionManager(listeningHost, listeningPort, &doThreadScgiConnection!(CustomCgi, fun, maxContentLength)); 3596 manager.listen(); 3597 } 3598 3599 /++ 3600 Serves a single "connection", but the connection is spoken on stdin and stdout instead of on a socket. 3601 3602 Intended for cases like working from systemd, like discussed here: https://forum.dlang.org/post/avmkfdiitirnrenzljwc@forum.dlang.org 3603 3604 History: 3605 Added May 29, 2021 3606 +/ 3607 void serveSingleHttpConnectionOnStdio(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3608 doThreadHttpConnectionGuts!(CustomCgi, fun, true)(new FakeSocketForStdin()); 3609 } 3610 3611 void stop() { 3612 // FIXME 3613 } 3614 } 3615 3616 private int privDropUserId; 3617 private int privDropGroupId; 3618 3619 // Added Jan 11, 2021 3620 private void dropPrivs() { 3621 version(Posix) { 3622 import core.sys.posix.unistd; 3623 3624 auto userId = privDropUserId; 3625 auto groupId = privDropGroupId; 3626 3627 if((userId != 0 || groupId != 0) && getuid() == 0) { 3628 if(groupId) 3629 setgid(groupId); 3630 if(userId) 3631 setuid(userId); 3632 } 3633 3634 } 3635 // FIXME: Windows? 3636 } 3637 3638 version(embedded_httpd_processes) 3639 void serveEmbeddedHttpdProcesses(alias fun, CustomCgi = Cgi)(RequestServer params) { 3640 import core.sys.posix.unistd; 3641 import core.sys.posix.sys.socket; 3642 import core.sys.posix.netinet.in_; 3643 //import std.c.linux.socket; 3644 3645 int sock = socket(AF_INET, SOCK_STREAM, 0); 3646 if(sock == -1) 3647 throw new Exception("socket"); 3648 3649 cloexec(sock); 3650 3651 { 3652 3653 sockaddr_in addr; 3654 addr.sin_family = AF_INET; 3655 addr.sin_port = htons(params.listeningPort); 3656 auto lh = params.listeningHost; 3657 if(lh.length) { 3658 if(inet_pton(AF_INET, lh.toStringz(), &addr.sin_addr.s_addr) != 1) 3659 throw new Exception("bad listening host given, please use an IP address.\nExample: --listening-host 127.0.0.1 means listen only on Localhost.\nExample: --listening-host 0.0.0.0 means listen on all interfaces.\nOr you can pass any other single numeric IPv4 address."); 3660 } else 3661 addr.sin_addr.s_addr = INADDR_ANY; 3662 3663 // HACKISH 3664 int on = 1; 3665 setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &on, on.sizeof); 3666 // end hack 3667 3668 3669 if(bind(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) { 3670 close(sock); 3671 throw new Exception("bind"); 3672 } 3673 3674 // FIXME: if this queue is full, it will just ignore it 3675 // and wait for the client to retransmit it. This is an 3676 // obnoxious timeout condition there. 3677 if(sock.listen(128) == -1) { 3678 close(sock); 3679 throw new Exception("listen"); 3680 } 3681 dropPrivs(); 3682 } 3683 3684 version(embedded_httpd_processes_accept_after_fork) {} else { 3685 int pipeReadFd; 3686 int pipeWriteFd; 3687 3688 { 3689 int[2] pipeFd; 3690 if(socketpair(AF_UNIX, SOCK_DGRAM, 0, pipeFd)) { 3691 import core.stdc.errno; 3692 throw new Exception("pipe failed " ~ to!string(errno)); 3693 } 3694 3695 pipeReadFd = pipeFd[0]; 3696 pipeWriteFd = pipeFd[1]; 3697 } 3698 } 3699 3700 3701 int processCount; 3702 pid_t newPid; 3703 reopen: 3704 while(processCount < processPoolSize) { 3705 newPid = fork(); 3706 if(newPid == 0) { 3707 // start serving on the socket 3708 //ubyte[4096] backingBuffer; 3709 for(;;) { 3710 bool closeConnection; 3711 uint i; 3712 sockaddr addr; 3713 i = addr.sizeof; 3714 version(embedded_httpd_processes_accept_after_fork) { 3715 int s = accept(sock, &addr, &i); 3716 int opt = 1; 3717 import core.sys.posix.netinet.tcp; 3718 // the Cgi class does internal buffering, so disabling this 3719 // helps with latency in many cases... 3720 setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &opt, opt.sizeof); 3721 cloexec(s); 3722 } else { 3723 int s; 3724 auto readret = read_fd(pipeReadFd, &s, s.sizeof, &s); 3725 if(readret != s.sizeof) { 3726 import core.stdc.errno; 3727 throw new Exception("pipe read failed " ~ to!string(errno)); 3728 } 3729 3730 //writeln("process ", getpid(), " got socket ", s); 3731 } 3732 3733 try { 3734 3735 if(s == -1) 3736 throw new Exception("accept"); 3737 3738 scope(failure) close(s); 3739 //ubyte[__traits(classInstanceSize, BufferedInputRange)] bufferedRangeContainer; 3740 auto ir = new BufferedInputRange(s); 3741 //auto ir = emplace!BufferedInputRange(bufferedRangeContainer, s, backingBuffer); 3742 3743 while(!ir.empty) { 3744 //ubyte[__traits(classInstanceSize, CustomCgi)] cgiContainer; 3745 3746 Cgi cgi; 3747 try { 3748 cgi = new CustomCgi(ir, &closeConnection); 3749 cgi._outputFileHandle = s; 3750 // if we have a single process and the browser tries to leave the connection open while concurrently requesting another, it will block everything an deadlock since there's no other server to accept it. By closing after each request in this situation, it tells the browser to serialize for us. 3751 if(processPoolSize <= 1) 3752 closeConnection = true; 3753 //cgi = emplace!CustomCgi(cgiContainer, ir, &closeConnection); 3754 } catch(Throwable t) { 3755 // a construction error is either bad code or bad request; bad request is what it should be since this is bug free :P 3756 // anyway let's kill the connection 3757 version(CRuntime_Musl) { 3758 // LockingTextWriter fails here 3759 // so working around it 3760 auto estr = t.toString(); 3761 stderr.rawWrite(estr); 3762 stderr.rawWrite("\n"); 3763 } else 3764 stderr.writeln(t.toString()); 3765 sendAll(ir.source, plainHttpError(false, "400 Bad Request", t)); 3766 closeConnection = true; 3767 break; 3768 } 3769 assert(cgi !is null); 3770 scope(exit) 3771 cgi.dispose(); 3772 3773 try { 3774 fun(cgi); 3775 cgi.close(); 3776 if(cgi.websocketMode) 3777 closeConnection = true; 3778 } catch(ConnectionException ce) { 3779 closeConnection = true; 3780 } catch(Throwable t) { 3781 // a processing error can be recovered from 3782 version(CRuntime_Musl) { 3783 // LockingTextWriter fails here 3784 // so working around it 3785 auto estr = t.toString(); 3786 stderr.rawWrite(estr); 3787 } else { 3788 stderr.writeln(t.toString); 3789 } 3790 if(!handleException(cgi, t)) 3791 closeConnection = true; 3792 } 3793 3794 if(closeConnection) { 3795 ir.source.close(); 3796 break; 3797 } else { 3798 if(!ir.empty) 3799 ir.popFront(); // get the next 3800 else if(ir.sourceClosed) { 3801 ir.source.close(); 3802 } 3803 } 3804 } 3805 3806 ir.source.close(); 3807 } catch(Throwable t) { 3808 version(CRuntime_Musl) {} else 3809 debug writeln(t); 3810 // most likely cause is a timeout 3811 } 3812 } 3813 } else if(newPid < 0) { 3814 throw new Exception("fork failed"); 3815 } else { 3816 processCount++; 3817 } 3818 } 3819 3820 // the parent should wait for its children... 3821 if(newPid) { 3822 import core.sys.posix.sys.wait; 3823 3824 version(embedded_httpd_processes_accept_after_fork) {} else { 3825 import core.sys.posix.sys.select; 3826 int[] fdQueue; 3827 while(true) { 3828 // writeln("select call"); 3829 int nfds = pipeWriteFd; 3830 if(sock > pipeWriteFd) 3831 nfds = sock; 3832 nfds += 1; 3833 fd_set read_fds; 3834 fd_set write_fds; 3835 FD_ZERO(&read_fds); 3836 FD_ZERO(&write_fds); 3837 FD_SET(sock, &read_fds); 3838 if(fdQueue.length) 3839 FD_SET(pipeWriteFd, &write_fds); 3840 auto ret = select(nfds, &read_fds, &write_fds, null, null); 3841 if(ret == -1) { 3842 import core.stdc.errno; 3843 if(errno == EINTR) 3844 goto try_wait; 3845 else 3846 throw new Exception("wtf select"); 3847 } 3848 3849 int s = -1; 3850 if(FD_ISSET(sock, &read_fds)) { 3851 uint i; 3852 sockaddr addr; 3853 i = addr.sizeof; 3854 s = accept(sock, &addr, &i); 3855 cloexec(s); 3856 import core.sys.posix.netinet.tcp; 3857 int opt = 1; 3858 setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &opt, opt.sizeof); 3859 } 3860 3861 if(FD_ISSET(pipeWriteFd, &write_fds)) { 3862 if(s == -1 && fdQueue.length) { 3863 s = fdQueue[0]; 3864 fdQueue = fdQueue[1 .. $]; // FIXME reuse buffer 3865 } 3866 write_fd(pipeWriteFd, &s, s.sizeof, s); 3867 close(s); // we are done with it, let the other process take ownership 3868 } else 3869 fdQueue ~= s; 3870 } 3871 } 3872 3873 try_wait: 3874 3875 int status; 3876 while(-1 != wait(&status)) { 3877 version(CRuntime_Musl) {} else { 3878 import std.stdio; writeln("Process died ", status); 3879 } 3880 processCount--; 3881 goto reopen; 3882 } 3883 close(sock); 3884 } 3885 } 3886 3887 version(fastcgi) 3888 void serveFastCgi(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)(RequestServer params) { 3889 // SetHandler fcgid-script 3890 FCGX_Stream* input, output, error; 3891 FCGX_ParamArray env; 3892 3893 3894 3895 const(ubyte)[] getFcgiChunk() { 3896 const(ubyte)[] ret; 3897 while(FCGX_HasSeenEOF(input) != -1) 3898 ret ~= cast(ubyte) FCGX_GetChar(input); 3899 return ret; 3900 } 3901 3902 void writeFcgi(const(ubyte)[] data) { 3903 FCGX_PutStr(data.ptr, data.length, output); 3904 } 3905 3906 void doARequest() { 3907 string[string] fcgienv; 3908 3909 for(auto e = env; e !is null && *e !is null; e++) { 3910 string cur = to!string(*e); 3911 auto idx = cur.indexOf("="); 3912 string name, value; 3913 if(idx == -1) 3914 name = cur; 3915 else { 3916 name = cur[0 .. idx]; 3917 value = cur[idx + 1 .. $]; 3918 } 3919 3920 fcgienv[name] = value; 3921 } 3922 3923 void flushFcgi() { 3924 FCGX_FFlush(output); 3925 } 3926 3927 Cgi cgi; 3928 try { 3929 cgi = new CustomCgi(maxContentLength, fcgienv, &getFcgiChunk, &writeFcgi, &flushFcgi); 3930 } catch(Throwable t) { 3931 FCGX_PutStr(cast(ubyte*) t.msg.ptr, t.msg.length, error); 3932 writeFcgi(cast(const(ubyte)[]) plainHttpError(true, "400 Bad Request", t)); 3933 return; //continue; 3934 } 3935 assert(cgi !is null); 3936 scope(exit) cgi.dispose(); 3937 try { 3938 fun(cgi); 3939 cgi.close(); 3940 } catch(Throwable t) { 3941 // log it to the error stream 3942 FCGX_PutStr(cast(ubyte*) t.msg.ptr, t.msg.length, error); 3943 // handle it for the user, if we can 3944 if(!handleException(cgi, t)) 3945 return; // continue; 3946 } 3947 } 3948 3949 auto lp = params.listeningPort; 3950 3951 FCGX_Request request; 3952 if(lp) { 3953 // if a listening port was specified on the command line, we want to spawn ourself 3954 // (needed for nginx without spawn-fcgi, e.g. on Windows) 3955 FCGX_Init(); 3956 auto sock = FCGX_OpenSocket(toStringz(params.listeningHost ~ ":" ~ to!string(lp)), 12); 3957 if(sock < 0) 3958 throw new Exception("Couldn't listen on the port"); 3959 FCGX_InitRequest(&request, sock, 0); 3960 while(FCGX_Accept_r(&request) >= 0) { 3961 input = request.inStream; 3962 output = request.outStream; 3963 error = request.errStream; 3964 env = request.envp; 3965 doARequest(); 3966 } 3967 } else { 3968 // otherwise, assume the httpd is doing it (the case for Apache, IIS, and Lighttpd) 3969 // using the version with a global variable since we are separate processes anyway 3970 while(FCGX_Accept(&input, &output, &error, &env) >= 0) { 3971 doARequest(); 3972 } 3973 } 3974 } 3975 3976 /// Returns the default listening port for the current cgi configuration. 8085 for embedded httpd, 4000 for scgi, irrelevant for others. 3977 ushort defaultListeningPort() { 3978 version(netman_httpd) 3979 return 8080; 3980 else version(embedded_httpd_processes) 3981 return 8085; 3982 else version(embedded_httpd_threads) 3983 return 8085; 3984 else version(scgi) 3985 return 4000; 3986 else 3987 return 0; 3988 } 3989 3990 /// Default host for listening. 127.0.0.1 for scgi, null (aka all interfaces) for all others. If you want the server directly accessible from other computers on the network, normally use null. If not, 127.0.0.1 is a bit better. Settable with default handlers with --listening-host command line argument. 3991 string defaultListeningHost() { 3992 version(netman_httpd) 3993 return null; 3994 else version(embedded_httpd_processes) 3995 return null; 3996 else version(embedded_httpd_threads) 3997 return null; 3998 else version(scgi) 3999 return "127.0.0.1"; 4000 else 4001 return null; 4002 4003 } 4004 4005 /++ 4006 This is the function [GenericMain] calls. View its source for some simple boilerplate you can copy/paste and modify, or you can call it yourself from your `main`. 4007 4008 Params: 4009 fun = Your request handler 4010 CustomCgi = a subclass of Cgi, if you wise to customize it further 4011 maxContentLength = max POST size you want to allow 4012 args = command-line arguments 4013 4014 History: 4015 Documented Sept 26, 2020. 4016 +/ 4017 void cgiMainImpl(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)(string[] args) if(is(CustomCgi : Cgi)) { 4018 if(tryAddonServers(args)) 4019 return; 4020 4021 if(trySimulatedRequest!(fun, CustomCgi)(args)) 4022 return; 4023 4024 RequestServer server; 4025 // you can change the port here if you like 4026 // server.listeningPort = 9000; 4027 4028 // then call this to let the command line args override your default 4029 server.configureFromCommandLine(args); 4030 4031 // and serve the request(s). 4032 server.serve!(fun, CustomCgi, maxContentLength)(); 4033 } 4034 4035 //version(plain_cgi) 4036 void handleCgiRequest(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 4037 // standard CGI is the default version 4038 Cgi cgi; 4039 try { 4040 cgi = new CustomCgi(maxContentLength); 4041 version(Posix) 4042 cgi._outputFileHandle = 1; // stdout 4043 else version(Windows) 4044 cgi._outputFileHandle = GetStdHandle(STD_OUTPUT_HANDLE); 4045 else static assert(0); 4046 } catch(Throwable t) { 4047 version(CRuntime_Musl) { 4048 // LockingTextWriter fails here 4049 // so working around it 4050 auto s = t.toString(); 4051 stderr.rawWrite(s); 4052 stdout.rawWrite(plainHttpError(true, "400 Bad Request", t)); 4053 } else { 4054 stderr.writeln(t.msg); 4055 // the real http server will probably handle this; 4056 // most likely, this is a bug in Cgi. But, oh well. 4057 stdout.write(plainHttpError(true, "400 Bad Request", t)); 4058 } 4059 return; 4060 } 4061 assert(cgi !is null); 4062 scope(exit) cgi.dispose(); 4063 4064 try { 4065 fun(cgi); 4066 cgi.close(); 4067 } catch (Throwable t) { 4068 version(CRuntime_Musl) { 4069 // LockingTextWriter fails here 4070 // so working around it 4071 auto s = t.msg; 4072 stderr.rawWrite(s); 4073 } else { 4074 stderr.writeln(t.msg); 4075 } 4076 if(!handleException(cgi, t)) 4077 return; 4078 } 4079 } 4080 4081 /+ 4082 The event loop for embedded_httpd_threads will prolly fiber dispatch 4083 cgi constructors too, so slow posts will not monopolize a worker thread. 4084 4085 May want to provide the worker task system just need to ensure all the fibers 4086 has a big enough stack for real work... would also ideally like to reuse them. 4087 4088 4089 So prolly bir would switch it to nonblocking. If it would block, it epoll 4090 registers one shot with this existing fiber to take it over. 4091 4092 new connection comes in. it picks a fiber off the free list, 4093 or if there is none, it creates a new one. this fiber handles 4094 this connection the whole time. 4095 4096 epoll triggers the fiber when something comes in. it is called by 4097 a random worker thread, it might change at any time. at least during 4098 the constructor. maybe into the main body it will stay tied to a thread 4099 just so TLS stuff doesn't randomly change in the middle. but I could 4100 specify if you yield all bets are off. 4101 4102 when the request is finished, if there's more data buffered, it just 4103 keeps going. if there is no more data buffered, it epoll ctls to 4104 get triggered when more data comes in. all one shot. 4105 4106 when a connection is closed, the fiber returns and is then reset 4107 and added to the free list. if the free list is full, the fiber is 4108 just freed, this means it will balloon to a certain size but not generally 4109 grow beyond that unless the activity keeps going. 4110 4111 256 KB stack i thnk per fiber. 4,000 active fibers per gigabyte of memory. 4112 4113 So the fiber has its own magic methods to read and write. if they would block, it registers 4114 for epoll and yields. when it returns, it read/writes and then returns back normal control. 4115 4116 basically you issue the command and it tells you when it is done 4117 4118 it needs to DEL the epoll thing when it is closed. add it when opened. mod it when anther thing issued 4119 4120 +/ 4121 4122 /++ 4123 The stack size when a fiber is created. You can set this from your main or from a shared static constructor 4124 to optimize your memory use if you know you don't need this much space. Be careful though, some functions use 4125 more stack space than you realize and a recursive function (including ones like in dom.d) can easily grow fast! 4126 4127 History: 4128 Added July 10, 2021. Previously, it used the druntime default of 16 KB. 4129 +/ 4130 version(cgi_use_fiber) 4131 __gshared size_t fiberStackSize = 4096 * 100; 4132 4133 version(cgi_use_fiber) 4134 class CgiFiber : Fiber { 4135 private void function(Socket) f_handler; 4136 private void f_handler_dg(Socket s) { // to avoid extra allocation w/ function 4137 f_handler(s); 4138 } 4139 this(void function(Socket) handler) { 4140 this.f_handler = handler; 4141 this(&f_handler_dg); 4142 } 4143 4144 this(void delegate(Socket) handler) { 4145 this.handler = handler; 4146 super(&run, fiberStackSize); 4147 } 4148 4149 Socket connection; 4150 void delegate(Socket) handler; 4151 4152 void run() { 4153 handler(connection); 4154 } 4155 4156 void delegate() postYield; 4157 4158 private void setPostYield(scope void delegate() py) @nogc { 4159 postYield = cast(void delegate()) py; 4160 } 4161 4162 void proceed() { 4163 try { 4164 call(); 4165 auto py = postYield; 4166 postYield = null; 4167 if(py !is null) 4168 py(); 4169 } catch(Exception e) { 4170 if(connection) 4171 connection.close(); 4172 goto terminate; 4173 } 4174 4175 if(state == State.TERM) { 4176 terminate: 4177 import core.memory; 4178 GC.removeRoot(cast(void*) this); 4179 } 4180 } 4181 } 4182 4183 void doThreadHttpConnection(CustomCgi, alias fun)(Socket connection) { 4184 assert(connection !is null); 4185 version(cgi_use_fiber) { 4186 auto fiber = new CgiFiber(&doThreadHttpConnectionGuts!(CustomCgi, fun)); 4187 import core.memory; 4188 GC.addRoot(cast(void*) fiber); 4189 fiber.connection = connection; 4190 fiber.proceed(); 4191 } else { 4192 doThreadHttpConnectionGuts!(CustomCgi, fun)(connection); 4193 } 4194 } 4195 4196 void doThreadHttpConnectionGuts(CustomCgi, alias fun, bool alwaysCloseConnection = false)(Socket connection) { 4197 scope(failure) { 4198 // catch all for other errors 4199 try { 4200 sendAll(connection, plainHttpError(false, "500 Internal Server Error", null)); 4201 connection.close(); 4202 } catch(Exception e) {} // swallow it, we're aborting anyway. 4203 } 4204 4205 bool closeConnection = alwaysCloseConnection; 4206 4207 /+ 4208 ubyte[4096] inputBuffer = void; 4209 ubyte[__traits(classInstanceSize, BufferedInputRange)] birBuffer = void; 4210 ubyte[__traits(classInstanceSize, CustomCgi)] cgiBuffer = void; 4211 4212 birBuffer[] = cast(ubyte[]) typeid(BufferedInputRange).initializer()[]; 4213 BufferedInputRange ir = cast(BufferedInputRange) cast(void*) birBuffer.ptr; 4214 ir.__ctor(connection, inputBuffer[], true); 4215 +/ 4216 4217 auto ir = new BufferedInputRange(connection); 4218 4219 while(!ir.empty) { 4220 4221 if(ir.view.length == 0) { 4222 ir.popFront(); 4223 if(ir.sourceClosed) { 4224 connection.close(); 4225 closeConnection = true; 4226 break; 4227 } 4228 } 4229 4230 Cgi cgi; 4231 try { 4232 cgi = new CustomCgi(ir, &closeConnection); 4233 cgi._outputFileHandle = connection.handle; 4234 } catch(ConnectionClosedException ce) { 4235 closeConnection = true; 4236 break; 4237 } catch(ConnectionException ce) { 4238 // broken pipe or something, just abort the connection 4239 closeConnection = true; 4240 break; 4241 } catch(Throwable t) { 4242 // a construction error is either bad code or bad request; bad request is what it should be since this is bug free :P 4243 // anyway let's kill the connection 4244 version(CRuntime_Musl) { 4245 stderr.rawWrite(t.toString()); 4246 stderr.rawWrite("\n"); 4247 } else { 4248 stderr.writeln(t.toString()); 4249 } 4250 sendAll(connection, plainHttpError(false, "400 Bad Request", t)); 4251 closeConnection = true; 4252 break; 4253 } 4254 assert(cgi !is null); 4255 scope(exit) 4256 cgi.dispose(); 4257 4258 try { 4259 fun(cgi); 4260 cgi.close(); 4261 if(cgi.websocketMode) 4262 closeConnection = true; 4263 } catch(ConnectionException ce) { 4264 // broken pipe or something, just abort the connection 4265 closeConnection = true; 4266 } catch(ConnectionClosedException ce) { 4267 // broken pipe or something, just abort the connection 4268 closeConnection = true; 4269 } catch(Throwable t) { 4270 // a processing error can be recovered from 4271 version(CRuntime_Musl) {} else 4272 stderr.writeln(t.toString); 4273 if(!handleException(cgi, t)) 4274 closeConnection = true; 4275 } 4276 4277 if(closeConnection || alwaysCloseConnection) { 4278 connection.shutdown(SocketShutdown.BOTH); 4279 connection.close(); 4280 ir.dispose(); 4281 closeConnection = false; // don't reclose after loop 4282 break; 4283 } else { 4284 if(ir.front.length) { 4285 ir.popFront(); // we can't just discard the buffer, so get the next bit and keep chugging along 4286 } else if(ir.sourceClosed) { 4287 ir.source.shutdown(SocketShutdown.BOTH); 4288 ir.source.close(); 4289 ir.dispose(); 4290 closeConnection = false; 4291 } else { 4292 continue; 4293 // break; // this was for a keepalive experiment 4294 } 4295 } 4296 } 4297 4298 if(closeConnection) { 4299 connection.shutdown(SocketShutdown.BOTH); 4300 connection.close(); 4301 ir.dispose(); 4302 } 4303 4304 // I am otherwise NOT closing it here because the parent thread might still be able to make use of the keep-alive connection! 4305 } 4306 4307 void doThreadScgiConnection(CustomCgi, alias fun, long maxContentLength)(Socket connection) { 4308 // and now we can buffer 4309 scope(failure) 4310 connection.close(); 4311 4312 import al = std.algorithm; 4313 4314 size_t size; 4315 4316 string[string] headers; 4317 4318 auto range = new BufferedInputRange(connection); 4319 more_data: 4320 auto chunk = range.front(); 4321 // waiting for colon for header length 4322 auto idx = indexOf(cast(string) chunk, ':'); 4323 if(idx == -1) { 4324 try { 4325 range.popFront(); 4326 } catch(Exception e) { 4327 // it is just closed, no big deal 4328 connection.close(); 4329 return; 4330 } 4331 goto more_data; 4332 } 4333 4334 size = to!size_t(cast(string) chunk[0 .. idx]); 4335 chunk = range.consume(idx + 1); 4336 // reading headers 4337 if(chunk.length < size) 4338 range.popFront(0, size + 1); 4339 // we are now guaranteed to have enough 4340 chunk = range.front(); 4341 assert(chunk.length > size); 4342 4343 idx = 0; 4344 string key; 4345 string value; 4346 foreach(part; al.splitter(chunk, '\0')) { 4347 if(idx & 1) { // odd is value 4348 value = cast(string)(part.idup); 4349 headers[key] = value; // commit 4350 } else 4351 key = cast(string)(part.idup); 4352 idx++; 4353 } 4354 4355 enforce(chunk[size] == ','); // the terminator 4356 4357 range.consume(size + 1); 4358 // reading data 4359 // this will be done by Cgi 4360 4361 const(ubyte)[] getScgiChunk() { 4362 // we are already primed 4363 auto data = range.front(); 4364 if(data.length == 0 && !range.sourceClosed) { 4365 range.popFront(0); 4366 data = range.front(); 4367 } else if (range.sourceClosed) 4368 range.source.close(); 4369 4370 return data; 4371 } 4372 4373 void writeScgi(const(ubyte)[] data) { 4374 sendAll(connection, data); 4375 } 4376 4377 void flushScgi() { 4378 // I don't *think* I have to do anything.... 4379 } 4380 4381 Cgi cgi; 4382 try { 4383 cgi = new CustomCgi(maxContentLength, headers, &getScgiChunk, &writeScgi, &flushScgi); 4384 cgi._outputFileHandle = connection.handle; 4385 } catch(Throwable t) { 4386 sendAll(connection, plainHttpError(true, "400 Bad Request", t)); 4387 connection.close(); 4388 return; // this connection is dead 4389 } 4390 assert(cgi !is null); 4391 scope(exit) cgi.dispose(); 4392 try { 4393 fun(cgi); 4394 cgi.close(); 4395 connection.close(); 4396 } catch(Throwable t) { 4397 // no std err 4398 if(!handleException(cgi, t)) { 4399 connection.close(); 4400 return; 4401 } else { 4402 connection.close(); 4403 return; 4404 } 4405 } 4406 } 4407 4408 string printDate(DateTime date) { 4409 char[29] buffer = void; 4410 printDateToBuffer(date, buffer[]); 4411 return buffer.idup; 4412 } 4413 4414 int printDateToBuffer(DateTime date, char[] buffer) @nogc { 4415 assert(buffer.length >= 29); 4416 // 29 static length ? 4417 4418 static immutable daysOfWeek = [ 4419 "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" 4420 ]; 4421 4422 static immutable months = [ 4423 null, "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" 4424 ]; 4425 4426 buffer[0 .. 3] = daysOfWeek[date.dayOfWeek]; 4427 buffer[3 .. 5] = ", "; 4428 buffer[5] = date.day / 10 + '0'; 4429 buffer[6] = date.day % 10 + '0'; 4430 buffer[7] = ' '; 4431 buffer[8 .. 11] = months[date.month]; 4432 buffer[11] = ' '; 4433 auto y = date.year; 4434 buffer[12] = cast(char) (y / 1000 + '0'); y %= 1000; 4435 buffer[13] = cast(char) (y / 100 + '0'); y %= 100; 4436 buffer[14] = cast(char) (y / 10 + '0'); y %= 10; 4437 buffer[15] = cast(char) (y + '0'); 4438 buffer[16] = ' '; 4439 buffer[17] = date.hour / 10 + '0'; 4440 buffer[18] = date.hour % 10 + '0'; 4441 buffer[19] = ':'; 4442 buffer[20] = date.minute / 10 + '0'; 4443 buffer[21] = date.minute % 10 + '0'; 4444 buffer[22] = ':'; 4445 buffer[23] = date.second / 10 + '0'; 4446 buffer[24] = date.second % 10 + '0'; 4447 buffer[25 .. $] = " GMT"; 4448 4449 return 29; 4450 } 4451 4452 4453 // Referencing this gigantic typeid seems to remind the compiler 4454 // to actually put the symbol in the object file. I guess the immutable 4455 // assoc array array isn't actually included in druntime 4456 void hackAroundLinkerError() { 4457 stdout.rawWrite(typeid(const(immutable(char)[][])[immutable(char)[]]).toString()); 4458 stdout.rawWrite(typeid(immutable(char)[][][immutable(char)[]]).toString()); 4459 stdout.rawWrite(typeid(Cgi.UploadedFile[immutable(char)[]]).toString()); 4460 stdout.rawWrite(typeid(Cgi.UploadedFile[][immutable(char)[]]).toString()); 4461 stdout.rawWrite(typeid(immutable(Cgi.UploadedFile)[immutable(char)[]]).toString()); 4462 stdout.rawWrite(typeid(immutable(Cgi.UploadedFile[])[immutable(char)[]]).toString()); 4463 stdout.rawWrite(typeid(immutable(char[])[immutable(char)[]]).toString()); 4464 // this is getting kinda ridiculous btw. Moving assoc arrays 4465 // to the library is the pain that keeps on coming. 4466 4467 // eh this broke the build on the work server 4468 // stdout.rawWrite(typeid(immutable(char)[][immutable(string[])])); 4469 stdout.rawWrite(typeid(immutable(string[])[immutable(char)[]]).toString()); 4470 } 4471 4472 4473 4474 4475 4476 version(fastcgi) { 4477 pragma(lib, "fcgi"); 4478 4479 static if(size_t.sizeof == 8) // 64 bit 4480 alias long c_int; 4481 else 4482 alias int c_int; 4483 4484 extern(C) { 4485 struct FCGX_Stream { 4486 ubyte* rdNext; 4487 ubyte* wrNext; 4488 ubyte* stop; 4489 ubyte* stopUnget; 4490 c_int isReader; 4491 c_int isClosed; 4492 c_int wasFCloseCalled; 4493 c_int FCGI_errno; 4494 void* function(FCGX_Stream* stream) fillBuffProc; 4495 void* function(FCGX_Stream* stream, c_int doClose) emptyBuffProc; 4496 void* data; 4497 } 4498 4499 // note: this is meant to be opaque, so don't access it directly 4500 struct FCGX_Request { 4501 int requestId; 4502 int role; 4503 FCGX_Stream* inStream; 4504 FCGX_Stream* outStream; 4505 FCGX_Stream* errStream; 4506 char** envp; 4507 void* paramsPtr; 4508 int ipcFd; 4509 int isBeginProcessed; 4510 int keepConnection; 4511 int appStatus; 4512 int nWriters; 4513 int flags; 4514 int listen_sock; 4515 } 4516 4517 int FCGX_InitRequest(FCGX_Request *request, int sock, int flags); 4518 void FCGX_Init(); 4519 4520 int FCGX_Accept_r(FCGX_Request *request); 4521 4522 4523 alias char** FCGX_ParamArray; 4524 4525 c_int FCGX_Accept(FCGX_Stream** stdin, FCGX_Stream** stdout, FCGX_Stream** stderr, FCGX_ParamArray* envp); 4526 c_int FCGX_GetChar(FCGX_Stream* stream); 4527 c_int FCGX_PutStr(const ubyte* str, c_int n, FCGX_Stream* stream); 4528 int FCGX_HasSeenEOF(FCGX_Stream* stream); 4529 c_int FCGX_FFlush(FCGX_Stream *stream); 4530 4531 int FCGX_OpenSocket(in char*, int); 4532 } 4533 } 4534 4535 4536 /* This might go int a separate module eventually. It is a network input helper class. */ 4537 4538 import std.socket; 4539 4540 version(cgi_use_fiber) { 4541 import core.thread; 4542 import core.sys.linux.epoll; 4543 4544 __gshared int epfd; 4545 } 4546 4547 4548 version(cgi_use_fiber) 4549 private enum WakeupEvent { 4550 Read = EPOLLIN, 4551 Write = EPOLLOUT 4552 } 4553 4554 version(cgi_use_fiber) 4555 private void registerEventWakeup(bool* registered, Socket source, WakeupEvent e) @nogc { 4556 4557 // static cast since I know what i have in here and don't want to pay for dynamic cast 4558 auto f = cast(CgiFiber) cast(void*) Fiber.getThis(); 4559 4560 f.setPostYield = () { 4561 if(*registered) { 4562 // rearm 4563 epoll_event evt; 4564 evt.events = e | EPOLLONESHOT; 4565 evt.data.ptr = cast(void*) f; 4566 if(epoll_ctl(epfd, EPOLL_CTL_MOD, source.handle, &evt) == -1) 4567 throw new Exception("epoll_ctl"); 4568 } else { 4569 // initial registration 4570 *registered = true ; 4571 int fd = source.handle; 4572 epoll_event evt; 4573 evt.events = e | EPOLLONESHOT; 4574 evt.data.ptr = cast(void*) f; 4575 if(epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &evt) == -1) 4576 throw new Exception("epoll_ctl"); 4577 } 4578 }; 4579 4580 Fiber.yield(); 4581 4582 f.setPostYield(null); 4583 } 4584 4585 version(cgi_use_fiber) 4586 void unregisterSource(Socket s) { 4587 epoll_event evt; 4588 epoll_ctl(epfd, EPOLL_CTL_DEL, s.handle(), &evt); 4589 } 4590 4591 // it is a class primarily for reference semantics 4592 // I might change this interface 4593 /// This is NOT ACTUALLY an input range! It is too different. Historical mistake kinda. 4594 class BufferedInputRange { 4595 version(Posix) 4596 this(int source, ubyte[] buffer = null) { 4597 this(new Socket(cast(socket_t) source, AddressFamily.INET), buffer); 4598 } 4599 4600 this(Socket source, ubyte[] buffer = null, bool allowGrowth = true) { 4601 // if they connect but never send stuff to us, we don't want it wasting the process 4602 // so setting a time out 4603 version(cgi_use_fiber) 4604 source.blocking = false; 4605 else 4606 source.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(3)); 4607 4608 this.source = source; 4609 if(buffer is null) { 4610 underlyingBuffer = new ubyte[4096]; 4611 this.allowGrowth = true; 4612 } else { 4613 underlyingBuffer = buffer; 4614 this.allowGrowth = allowGrowth; 4615 } 4616 4617 assert(underlyingBuffer.length); 4618 4619 // we assume view.ptr is always inside underlyingBuffer 4620 view = underlyingBuffer[0 .. 0]; 4621 4622 popFront(); // prime 4623 } 4624 4625 version(cgi_use_fiber) { 4626 bool registered; 4627 } 4628 4629 void dispose() { 4630 version(cgi_use_fiber) { 4631 if(registered) 4632 unregisterSource(source); 4633 } 4634 } 4635 4636 /** 4637 A slight difference from regular ranges is you can give it the maximum 4638 number of bytes to consume. 4639 4640 IMPORTANT NOTE: the default is to consume nothing, so if you don't call 4641 consume() yourself and use a regular foreach, it will infinitely loop! 4642 4643 The default is to do what a normal range does, and consume the whole buffer 4644 and wait for additional input. 4645 4646 You can also specify 0, to append to the buffer, or any other number 4647 to remove the front n bytes and wait for more. 4648 */ 4649 void popFront(size_t maxBytesToConsume = 0 /*size_t.max*/, size_t minBytesToSettleFor = 0, bool skipConsume = false) { 4650 if(sourceClosed) 4651 throw new ConnectionClosedException("can't get any more data from a closed source"); 4652 if(!skipConsume) 4653 consume(maxBytesToConsume); 4654 4655 // we might have to grow the buffer 4656 if(minBytesToSettleFor > underlyingBuffer.length || view.length == underlyingBuffer.length) { 4657 if(allowGrowth) { 4658 //import std.stdio; writeln("growth"); 4659 auto viewStart = view.ptr - underlyingBuffer.ptr; 4660 size_t growth = 4096; 4661 // make sure we have enough for what we're being asked for 4662 if(minBytesToSettleFor > 0 && minBytesToSettleFor - underlyingBuffer.length > growth) 4663 growth = minBytesToSettleFor - underlyingBuffer.length; 4664 //import std.stdio; writeln(underlyingBuffer.length, " ", viewStart, " ", view.length, " ", growth, " ", minBytesToSettleFor, " ", minBytesToSettleFor - underlyingBuffer.length); 4665 underlyingBuffer.length += growth; 4666 view = underlyingBuffer[viewStart .. view.length]; 4667 } else 4668 throw new Exception("No room left in the buffer"); 4669 } 4670 4671 do { 4672 auto freeSpace = underlyingBuffer[view.ptr - underlyingBuffer.ptr + view.length .. $]; 4673 try_again: 4674 auto ret = source.receive(freeSpace); 4675 if(ret == Socket.ERROR) { 4676 if(wouldHaveBlocked()) { 4677 version(cgi_use_fiber) { 4678 registerEventWakeup(®istered, source, WakeupEvent.Read); 4679 goto try_again; 4680 } else { 4681 // gonna treat a timeout here as a close 4682 sourceClosed = true; 4683 return; 4684 } 4685 } 4686 version(Posix) { 4687 import core.stdc.errno; 4688 if(errno == EINTR || errno == EAGAIN) { 4689 goto try_again; 4690 } 4691 if(errno == ECONNRESET) { 4692 sourceClosed = true; 4693 return; 4694 } 4695 } 4696 throw new Exception(lastSocketError); // FIXME 4697 } 4698 if(ret == 0) { 4699 sourceClosed = true; 4700 return; 4701 } 4702 4703 //import std.stdio; writeln(view.ptr); writeln(underlyingBuffer.ptr); writeln(view.length, " ", ret, " = ", view.length + ret); 4704 view = underlyingBuffer[view.ptr - underlyingBuffer.ptr .. view.length + ret]; 4705 //import std.stdio; writeln(cast(string) view); 4706 } while(view.length < minBytesToSettleFor); 4707 } 4708 4709 /// Removes n bytes from the front of the buffer, and returns the new buffer slice. 4710 /// You might want to idup the data you are consuming if you store it, since it may 4711 /// be overwritten on the new popFront. 4712 /// 4713 /// You do not need to call this if you always want to wait for more data when you 4714 /// consume some. 4715 ubyte[] consume(size_t bytes) { 4716 //import std.stdio; writeln("consuime ", bytes, "/", view.length); 4717 view = view[bytes > $ ? $ : bytes .. $]; 4718 if(view.length == 0) { 4719 view = underlyingBuffer[0 .. 0]; // go ahead and reuse the beginning 4720 /* 4721 writeln("HERE"); 4722 popFront(0, 0, true); // try to load more if we can, checks if the source is closed 4723 writeln(cast(string)front); 4724 writeln("DONE"); 4725 */ 4726 } 4727 return front; 4728 } 4729 4730 bool empty() { 4731 return sourceClosed && view.length == 0; 4732 } 4733 4734 ubyte[] front() { 4735 return view; 4736 } 4737 4738 invariant() { 4739 assert(view.ptr >= underlyingBuffer.ptr); 4740 // it should never be equal, since if that happens view ought to be empty, and thus reusing the buffer 4741 assert(view.ptr < underlyingBuffer.ptr + underlyingBuffer.length); 4742 } 4743 4744 ubyte[] underlyingBuffer; 4745 bool allowGrowth; 4746 ubyte[] view; 4747 Socket source; 4748 bool sourceClosed; 4749 } 4750 4751 private class FakeSocketForStdin : Socket { 4752 import std.stdio; 4753 4754 this() { 4755 4756 } 4757 4758 private bool closed; 4759 4760 override ptrdiff_t receive(void[] buffer, std.socket.SocketFlags) @trusted { 4761 if(closed) 4762 throw new Exception("Closed"); 4763 return stdin.rawRead(buffer).length; 4764 } 4765 4766 override ptrdiff_t send(const void[] buffer, std.socket.SocketFlags) @trusted { 4767 if(closed) 4768 throw new Exception("Closed"); 4769 stdout.rawWrite(buffer); 4770 return buffer.length; 4771 } 4772 4773 override void close() @trusted { 4774 (cast(void delegate() @nogc nothrow) &realClose)(); 4775 } 4776 4777 override void shutdown(SocketShutdown s) { 4778 // FIXME 4779 } 4780 4781 override void setOption(SocketOptionLevel, SocketOption, void[]) {} 4782 override void setOption(SocketOptionLevel, SocketOption, Duration) {} 4783 4784 override @property @trusted Address remoteAddress() { return null; } 4785 override @property @trusted Address localAddress() { return null; } 4786 4787 void realClose() { 4788 closed = true; 4789 try { 4790 stdin.close(); 4791 stdout.close(); 4792 } catch(Exception e) { 4793 4794 } 4795 } 4796 } 4797 4798 import core.sync.semaphore; 4799 import core.atomic; 4800 4801 /** 4802 To use this thing: 4803 4804 void handler(Socket s) { do something... } 4805 auto manager = new ListeningConnectionManager("127.0.0.1", 80, &handler); 4806 manager.listen(); 4807 4808 I suggest you use BufferedInputRange(connection) to handle the input. As a packet 4809 comes in, you will get control. You can just continue; though to fetch more. 4810 4811 4812 FIXME: should I offer an event based async thing like netman did too? Yeah, probably. 4813 */ 4814 class ListeningConnectionManager { 4815 Semaphore semaphore; 4816 Socket[256] queue; 4817 shared(ubyte) nextIndexFront; 4818 ubyte nextIndexBack; 4819 shared(int) queueLength; 4820 4821 void listen() { 4822 running = true; 4823 shared(int) loopBroken; 4824 4825 version(Posix) { 4826 import core.sys.posix.signal; 4827 signal(SIGPIPE, SIG_IGN); 4828 } 4829 4830 version(cgi_no_threads) { 4831 // NEVER USE THIS 4832 // it exists only for debugging and other special occasions 4833 4834 // the thread mode is faster and less likely to stall the whole 4835 // thing when a request is slow 4836 while(!loopBroken && running) { 4837 auto sn = listener.accept(); 4838 cloexec(sn); 4839 try { 4840 handler(sn); 4841 } catch(Exception e) { 4842 // if a connection goes wrong, we want to just say no, but try to carry on unless it is an Error of some sort (in which case, we'll die. You might want an external helper program to revive the server when it dies) 4843 sn.close(); 4844 } 4845 } 4846 } else { 4847 import std.parallelism; 4848 4849 version(cgi_use_fork) { 4850 //asm { int 3; } 4851 fork(); 4852 } 4853 4854 version(cgi_use_fiber) { 4855 import core.sys.linux.epoll; 4856 epfd = epoll_create1(EPOLL_CLOEXEC); 4857 if(epfd == -1) 4858 throw new Exception("epoll_create1 " ~ to!string(errno)); 4859 scope(exit) { 4860 import core.sys.posix.unistd; 4861 close(epfd); 4862 } 4863 4864 epoll_event ev; 4865 ev.events = EPOLLIN | EPOLLEXCLUSIVE; // EPOLLEXCLUSIVE is only available on kernels since like 2017 but that's prolly good enough. 4866 ev.data.fd = listener.handle; 4867 if(epoll_ctl(epfd, EPOLL_CTL_ADD, listener.handle, &ev) == -1) 4868 throw new Exception("epoll_ctl " ~ to!string(errno)); 4869 4870 WorkerThread[] threads = new WorkerThread[](totalCPUs * 1 + 1); 4871 foreach(i, ref thread; threads) { 4872 thread = new WorkerThread(this, handler, cast(int) i); 4873 thread.start(); 4874 } 4875 4876 bool fiber_crash_check() { 4877 bool hasAnyRunning; 4878 foreach(thread; threads) { 4879 if(!thread.isRunning) { 4880 thread.join(); 4881 } else hasAnyRunning = true; 4882 } 4883 4884 return (!hasAnyRunning); 4885 } 4886 4887 4888 while(running) { 4889 Thread.sleep(1.seconds); 4890 if(fiber_crash_check()) 4891 break; 4892 } 4893 4894 } else { 4895 semaphore = new Semaphore(); 4896 4897 // I times 4 here because there's a good chance some will be blocked on i/o. 4898 ConnectionThread[] threads = new ConnectionThread[](totalCPUs * 4); 4899 foreach(i, ref thread; threads) { 4900 thread = new ConnectionThread(this, handler, cast(int) i); 4901 thread.start(); 4902 } 4903 } 4904 4905 while(!loopBroken && running) { 4906 Socket sn; 4907 4908 bool crash_check() { 4909 bool hasAnyRunning; 4910 foreach(thread; threads) { 4911 if(!thread.isRunning) { 4912 thread.join(); 4913 } else hasAnyRunning = true; 4914 } 4915 4916 return (!hasAnyRunning); 4917 } 4918 4919 4920 void accept_new_connection() { 4921 sn = listener.accept(); 4922 cloexec(sn); 4923 if(tcp) { 4924 // disable Nagle's algorithm to avoid a 40ms delay when we send/recv 4925 // on the socket because we do some buffering internally. I think this helps, 4926 // certainly does for small requests, and I think it does for larger ones too 4927 sn.setOption(SocketOptionLevel.TCP, SocketOption.TCP_NODELAY, 1); 4928 4929 sn.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(10)); 4930 } 4931 } 4932 4933 void existing_connection_new_data() { 4934 // wait until a slot opens up 4935 //int waited = 0; 4936 while(queueLength >= queue.length) { 4937 Thread.sleep(1.msecs); 4938 //waited ++; 4939 } 4940 //if(waited) {import std.stdio; writeln(waited);} 4941 synchronized(this) { 4942 queue[nextIndexBack] = sn; 4943 nextIndexBack++; 4944 atomicOp!"+="(queueLength, 1); 4945 } 4946 semaphore.notify(); 4947 } 4948 4949 4950 accept_new_connection(); 4951 existing_connection_new_data(); 4952 4953 if(crash_check()) 4954 break; 4955 } 4956 4957 // FIXME: i typically stop this with ctrl+c which never 4958 // actually gets here. i need to do a sigint handler. 4959 if(cleanup) 4960 cleanup(); 4961 } 4962 } 4963 4964 //version(linux) 4965 //int epoll_fd; 4966 4967 bool tcp; 4968 void delegate() cleanup; 4969 4970 private void function(Socket) fhandler; 4971 private void dg_handler(Socket s) { 4972 fhandler(s); 4973 } 4974 this(string host, ushort port, void function(Socket) handler) { 4975 fhandler = handler; 4976 this(host, port, &dg_handler); 4977 } 4978 4979 this(string host, ushort port, void delegate(Socket) handler) { 4980 this.handler = handler; 4981 4982 listener = startListening(host, port, tcp, cleanup, 128); 4983 4984 version(cgi_use_fiber) version(cgi_use_fork) 4985 listener.blocking = false; 4986 4987 // this is the UI control thread and thus gets more priority 4988 Thread.getThis.priority = Thread.PRIORITY_MAX; 4989 } 4990 4991 Socket listener; 4992 void delegate(Socket) handler; 4993 4994 bool running; 4995 void quit() { 4996 running = false; 4997 } 4998 } 4999 5000 Socket startListening(string host, ushort port, ref bool tcp, ref void delegate() cleanup, int backQueue) { 5001 Socket listener; 5002 if(host.startsWith("unix:")) { 5003 version(Posix) { 5004 listener = new Socket(AddressFamily.UNIX, SocketType.STREAM); 5005 cloexec(listener); 5006 string filename = host["unix:".length .. $].idup; 5007 listener.bind(new UnixAddress(filename)); 5008 cleanup = delegate() { 5009 listener.close(); 5010 import std.file; 5011 remove(filename); 5012 }; 5013 tcp = false; 5014 } else { 5015 throw new Exception("unix sockets not supported on this system"); 5016 } 5017 } else if(host.startsWith("abstract:")) { 5018 version(linux) { 5019 listener = new Socket(AddressFamily.UNIX, SocketType.STREAM); 5020 cloexec(listener); 5021 string filename = "\0" ~ host["abstract:".length .. $]; 5022 import std.stdio; stderr.writeln("Listening to abstract unix domain socket: ", host["abstract:".length .. $]); 5023 listener.bind(new UnixAddress(filename)); 5024 tcp = false; 5025 } else { 5026 throw new Exception("abstract unix sockets not supported on this system"); 5027 } 5028 } else { 5029 listener = new TcpSocket(); 5030 cloexec(listener); 5031 listener.setOption(SocketOptionLevel.SOCKET, SocketOption.REUSEADDR, true); 5032 listener.bind(host.length ? parseAddress(host, port) : new InternetAddress(port)); 5033 cleanup = delegate() { 5034 listener.close(); 5035 }; 5036 tcp = true; 5037 } 5038 5039 listener.listen(backQueue); 5040 5041 dropPrivs(); 5042 5043 return listener; 5044 } 5045 5046 // helper function to send a lot to a socket. Since this blocks for the buffer (possibly several times), you should probably call it in a separate thread or something. 5047 void sendAll(Socket s, const(void)[] data, string file = __FILE__, size_t line = __LINE__) { 5048 if(data.length == 0) return; 5049 ptrdiff_t amount; 5050 //import std.stdio; writeln("***",cast(string) data,"///"); 5051 do { 5052 amount = s.send(data); 5053 if(amount == Socket.ERROR) { 5054 version(cgi_use_fiber) { 5055 if(wouldHaveBlocked()) { 5056 bool registered = true; 5057 registerEventWakeup(®istered, s, WakeupEvent.Write); 5058 continue; 5059 } 5060 } 5061 throw new ConnectionException(s, lastSocketError, file, line); 5062 } 5063 assert(amount > 0); 5064 5065 data = data[amount .. $]; 5066 } while(data.length); 5067 } 5068 5069 class ConnectionException : Exception { 5070 Socket socket; 5071 this(Socket s, string msg, string file = __FILE__, size_t line = __LINE__) { 5072 this.socket = s; 5073 super(msg, file, line); 5074 } 5075 } 5076 5077 alias void delegate(Socket) CMT; 5078 5079 import core.thread; 5080 /+ 5081 cgi.d now uses a hybrid of event i/o and threads at the top level. 5082 5083 Top level thread is responsible for accepting sockets and selecting on them. 5084 5085 It then indicates to a child that a request is pending, and any random worker 5086 thread that is free handles it. It goes into blocking mode and handles that 5087 http request to completion. 5088 5089 At that point, it goes back into the waiting queue. 5090 5091 5092 This concept is only implemented on Linux. On all other systems, it still 5093 uses the worker threads and semaphores (which is perfectly fine for a lot of 5094 things! Just having a great number of keep-alive connections will break that.) 5095 5096 5097 So the algorithm is: 5098 5099 select(accept, event, pending) 5100 if accept -> send socket to free thread, if any. if not, add socket to queue 5101 if event -> send the signaling thread a socket from the queue, if not, mark it free 5102 - event might block until it can be *written* to. it is a fifo sending socket fds! 5103 5104 A worker only does one http request at a time, then signals its availability back to the boss. 5105 5106 The socket the worker was just doing should be added to the one-off epoll read. If it is closed, 5107 great, we can get rid of it. Otherwise, it is considered `pending`. The *kernel* manages that; the 5108 actual FD will not be kept out here. 5109 5110 So: 5111 queue = sockets we know are ready to read now, but no worker thread is available 5112 idle list = worker threads not doing anything else. they signal back and forth 5113 5114 the workers all read off the event fd. This is the semaphore wait 5115 5116 the boss waits on accept or other sockets read events (one off! and level triggered). If anything happens wrt ready read, 5117 it puts it in the queue and writes to the event fd. 5118 5119 The child could put the socket back in the epoll thing itself. 5120 5121 The child needs to be able to gracefully handle being given a socket that just closed with no work. 5122 +/ 5123 class ConnectionThread : Thread { 5124 this(ListeningConnectionManager lcm, CMT dg, int myThreadNumber) { 5125 this.lcm = lcm; 5126 this.dg = dg; 5127 this.myThreadNumber = myThreadNumber; 5128 super(&run); 5129 } 5130 5131 void run() { 5132 while(true) { 5133 // so if there's a bunch of idle keep-alive connections, it can 5134 // consume all the worker threads... just sitting there. 5135 lcm.semaphore.wait(); 5136 Socket socket; 5137 synchronized(lcm) { 5138 auto idx = lcm.nextIndexFront; 5139 socket = lcm.queue[idx]; 5140 lcm.queue[idx] = null; 5141 atomicOp!"+="(lcm.nextIndexFront, 1); 5142 atomicOp!"-="(lcm.queueLength, 1); 5143 } 5144 try { 5145 //import std.stdio; writeln(myThreadNumber, " taking it"); 5146 dg(socket); 5147 /+ 5148 if(socket.isAlive) { 5149 // process it more later 5150 version(linux) { 5151 import core.sys.linux.epoll; 5152 epoll_event ev; 5153 ev.events = EPOLLIN | EPOLLONESHOT | EPOLLET; 5154 ev.data.fd = socket.handle; 5155 import std.stdio; writeln("adding"); 5156 if(epoll_ctl(lcm.epoll_fd, EPOLL_CTL_ADD, socket.handle, &ev) == -1) { 5157 if(errno == EEXIST) { 5158 ev.events = EPOLLIN | EPOLLONESHOT | EPOLLET; 5159 ev.data.fd = socket.handle; 5160 if(epoll_ctl(lcm.epoll_fd, EPOLL_CTL_MOD, socket.handle, &ev) == -1) 5161 throw new Exception("epoll_ctl " ~ to!string(errno)); 5162 } else 5163 throw new Exception("epoll_ctl " ~ to!string(errno)); 5164 } 5165 //import std.stdio; writeln("keep alive"); 5166 // writing to this private member is to prevent the GC from closing my precious socket when I'm trying to use it later 5167 __traits(getMember, socket, "sock") = cast(socket_t) -1; 5168 } else { 5169 continue; // hope it times out in a reasonable amount of time... 5170 } 5171 } 5172 +/ 5173 } catch(ConnectionClosedException e) { 5174 // can just ignore this, it is fairly normal 5175 socket.close(); 5176 } catch(Throwable e) { 5177 import std.stdio; stderr.rawWrite(e.toString); stderr.rawWrite("\n"); 5178 socket.close(); 5179 } 5180 } 5181 } 5182 5183 ListeningConnectionManager lcm; 5184 CMT dg; 5185 int myThreadNumber; 5186 } 5187 5188 version(cgi_use_fiber) 5189 class WorkerThread : Thread { 5190 this(ListeningConnectionManager lcm, CMT dg, int myThreadNumber) { 5191 this.lcm = lcm; 5192 this.dg = dg; 5193 this.myThreadNumber = myThreadNumber; 5194 super(&run); 5195 } 5196 5197 void run() { 5198 while(lcm.running) { 5199 Socket sn; 5200 5201 epoll_event[64] events; 5202 auto nfds = epoll_wait(epfd, events.ptr, events.length, -1); 5203 if(nfds == -1) { 5204 if(errno == EINTR) 5205 continue; 5206 throw new Exception("epoll_wait " ~ to!string(errno)); 5207 } 5208 5209 foreach(idx; 0 .. nfds) { 5210 auto flags = events[idx].events; 5211 5212 if(cast(size_t) events[idx].data.ptr == cast(size_t) lcm.listener.handle) { 5213 // this try/catch is because it is set to non-blocking mode 5214 // and Phobos' stupid api throws an exception instead of returning 5215 // if it would block. Why would it block? because a forked process 5216 // might have beat us to it, but the wakeup event thundered our herds. 5217 version(cgi_use_fork) { 5218 try 5219 sn = lcm.listener.accept(); 5220 catch(SocketAcceptException e) { continue; } 5221 } else { 5222 sn = lcm.listener.accept(); 5223 } 5224 5225 cloexec(sn); 5226 if(lcm.tcp) { 5227 // disable Nagle's algorithm to avoid a 40ms delay when we send/recv 5228 // on the socket because we do some buffering internally. I think this helps, 5229 // certainly does for small requests, and I think it does for larger ones too 5230 sn.setOption(SocketOptionLevel.TCP, SocketOption.TCP_NODELAY, 1); 5231 5232 sn.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(10)); 5233 } 5234 5235 dg(sn); 5236 } else { 5237 auto fiber = cast(CgiFiber) events[idx].data.ptr; 5238 fiber.proceed(); 5239 } 5240 } 5241 } 5242 } 5243 5244 ListeningConnectionManager lcm; 5245 CMT dg; 5246 int myThreadNumber; 5247 } 5248 5249 5250 /* Done with network helper */ 5251 5252 /* Helpers for doing temporary files. Used both here and in web.d */ 5253 5254 version(Windows) { 5255 import core.sys.windows.windows; 5256 extern(Windows) DWORD GetTempPathW(DWORD, LPWSTR); 5257 alias GetTempPathW GetTempPath; 5258 } 5259 5260 version(Posix) { 5261 static import linux = core.sys.posix.unistd; 5262 } 5263 5264 string getTempDirectory() { 5265 string path; 5266 version(Windows) { 5267 wchar[1024] buffer; 5268 auto len = GetTempPath(1024, buffer.ptr); 5269 if(len == 0) 5270 throw new Exception("couldn't find a temporary path"); 5271 5272 auto b = buffer[0 .. len]; 5273 5274 path = to!string(b); 5275 } else 5276 path = "/tmp/"; 5277 5278 return path; 5279 } 5280 5281 5282 // I like std.date. These functions help keep my old code and data working with phobos changing. 5283 5284 long sysTimeToDTime(in SysTime sysTime) { 5285 return convert!("hnsecs", "msecs")(sysTime.stdTime - 621355968000000000L); 5286 } 5287 5288 long dateTimeToDTime(in DateTime dt) { 5289 return sysTimeToDTime(cast(SysTime) dt); 5290 } 5291 5292 long getUtcTime() { // renamed primarily to avoid conflict with std.date itself 5293 return sysTimeToDTime(Clock.currTime(UTC())); 5294 } 5295 5296 // NOTE: new SimpleTimeZone(minutes); can perhaps work with the getTimezoneOffset() JS trick 5297 SysTime dTimeToSysTime(long dTime, immutable TimeZone tz = null) { 5298 immutable hnsecs = convert!("msecs", "hnsecs")(dTime) + 621355968000000000L; 5299 return SysTime(hnsecs, tz); 5300 } 5301 5302 5303 5304 // this is a helper to read HTTP transfer-encoding: chunked responses 5305 immutable(ubyte[]) dechunk(BufferedInputRange ir) { 5306 immutable(ubyte)[] ret; 5307 5308 another_chunk: 5309 // If here, we are at the beginning of a chunk. 5310 auto a = ir.front(); 5311 int chunkSize; 5312 int loc = locationOf(a, "\r\n"); 5313 while(loc == -1) { 5314 ir.popFront(); 5315 a = ir.front(); 5316 loc = locationOf(a, "\r\n"); 5317 } 5318 5319 string hex; 5320 hex = ""; 5321 for(int i = 0; i < loc; i++) { 5322 char c = a[i]; 5323 if(c >= 'A' && c <= 'Z') 5324 c += 0x20; 5325 if((c >= '0' && c <= '9') || (c >= 'a' && c <= 'z')) { 5326 hex ~= c; 5327 } else { 5328 break; 5329 } 5330 } 5331 5332 assert(hex.length); 5333 5334 int power = 1; 5335 int size = 0; 5336 foreach(cc1; retro(hex)) { 5337 dchar cc = cc1; 5338 if(cc >= 'a' && cc <= 'z') 5339 cc -= 0x20; 5340 int val = 0; 5341 if(cc >= '0' && cc <= '9') 5342 val = cc - '0'; 5343 else 5344 val = cc - 'A' + 10; 5345 5346 size += power * val; 5347 power *= 16; 5348 } 5349 5350 chunkSize = size; 5351 assert(size >= 0); 5352 5353 if(loc + 2 > a.length) { 5354 ir.popFront(0, a.length + loc + 2); 5355 a = ir.front(); 5356 } 5357 5358 a = ir.consume(loc + 2); 5359 5360 if(chunkSize == 0) { // we're done with the response 5361 // if we got here, will change must be true.... 5362 more_footers: 5363 loc = locationOf(a, "\r\n"); 5364 if(loc == -1) { 5365 ir.popFront(); 5366 a = ir.front; 5367 goto more_footers; 5368 } else { 5369 assert(loc == 0); 5370 ir.consume(loc + 2); 5371 goto finish; 5372 } 5373 } else { 5374 // if we got here, will change must be true.... 5375 if(a.length < chunkSize + 2) { 5376 ir.popFront(0, chunkSize + 2); 5377 a = ir.front(); 5378 } 5379 5380 ret ~= (a[0..chunkSize]); 5381 5382 if(!(a.length > chunkSize + 2)) { 5383 ir.popFront(0, chunkSize + 2); 5384 a = ir.front(); 5385 } 5386 assert(a[chunkSize] == 13); 5387 assert(a[chunkSize+1] == 10); 5388 a = ir.consume(chunkSize + 2); 5389 chunkSize = 0; 5390 goto another_chunk; 5391 } 5392 5393 finish: 5394 return ret; 5395 } 5396 5397 // I want to be able to get data from multiple sources the same way... 5398 interface ByChunkRange { 5399 bool empty(); 5400 void popFront(); 5401 const(ubyte)[] front(); 5402 } 5403 5404 ByChunkRange byChunk(const(ubyte)[] data) { 5405 return new class ByChunkRange { 5406 override bool empty() { 5407 return !data.length; 5408 } 5409 5410 override void popFront() { 5411 if(data.length > 4096) 5412 data = data[4096 .. $]; 5413 else 5414 data = null; 5415 } 5416 5417 override const(ubyte)[] front() { 5418 return data[0 .. $ > 4096 ? 4096 : $]; 5419 } 5420 }; 5421 } 5422 5423 ByChunkRange byChunk(BufferedInputRange ir, size_t atMost) { 5424 const(ubyte)[] f; 5425 5426 f = ir.front; 5427 if(f.length > atMost) 5428 f = f[0 .. atMost]; 5429 5430 return new class ByChunkRange { 5431 override bool empty() { 5432 return atMost == 0; 5433 } 5434 5435 override const(ubyte)[] front() { 5436 return f; 5437 } 5438 5439 override void popFront() { 5440 ir.consume(f.length); 5441 atMost -= f.length; 5442 auto a = ir.front(); 5443 5444 if(a.length <= atMost) { 5445 f = a; 5446 atMost -= a.length; 5447 a = ir.consume(a.length); 5448 if(atMost != 0) 5449 ir.popFront(); 5450 if(f.length == 0) { 5451 f = ir.front(); 5452 } 5453 } else { 5454 // we actually have *more* here than we need.... 5455 f = a[0..atMost]; 5456 atMost = 0; 5457 ir.consume(atMost); 5458 } 5459 } 5460 }; 5461 } 5462 5463 version(cgi_with_websocket) { 5464 // http://tools.ietf.org/html/rfc6455 5465 5466 /** 5467 WEBSOCKET SUPPORT: 5468 5469 Full example: 5470 --- 5471 import arsd.cgi; 5472 5473 void websocketEcho(Cgi cgi) { 5474 if(cgi.websocketRequested()) { 5475 if(cgi.origin != "http://arsdnet.net") 5476 throw new Exception("bad origin"); 5477 auto websocket = cgi.acceptWebsocket(); 5478 5479 websocket.send("hello"); 5480 websocket.send(" world!"); 5481 5482 auto msg = websocket.recv(); 5483 while(msg.opcode != WebSocketOpcode.close) { 5484 if(msg.opcode == WebSocketOpcode.text) { 5485 websocket.send(msg.textData); 5486 } else if(msg.opcode == WebSocketOpcode.binary) { 5487 websocket.send(msg.data); 5488 } 5489 5490 msg = websocket.recv(); 5491 } 5492 5493 websocket.close(); 5494 } else assert(0, "i want a web socket!"); 5495 } 5496 5497 mixin GenericMain!websocketEcho; 5498 --- 5499 */ 5500 5501 class WebSocket { 5502 Cgi cgi; 5503 5504 private this(Cgi cgi) { 5505 this.cgi = cgi; 5506 5507 Socket socket = cgi.idlol.source; 5508 socket.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"minutes"(5)); 5509 } 5510 5511 // returns true if data available, false if it timed out 5512 bool recvAvailable(Duration timeout = dur!"msecs"(0)) { 5513 if(!waitForNextMessageWouldBlock()) 5514 return true; 5515 if(isDataPending(timeout)) 5516 return true; // this is kinda a lie. 5517 5518 return false; 5519 } 5520 5521 public bool lowLevelReceive() { 5522 auto bfr = cgi.idlol; 5523 top: 5524 auto got = bfr.front; 5525 if(got.length) { 5526 if(receiveBuffer.length < receiveBufferUsedLength + got.length) 5527 receiveBuffer.length += receiveBufferUsedLength + got.length; 5528 5529 receiveBuffer[receiveBufferUsedLength .. receiveBufferUsedLength + got.length] = got[]; 5530 receiveBufferUsedLength += got.length; 5531 bfr.consume(got.length); 5532 5533 return true; 5534 } 5535 5536 if(bfr.sourceClosed) 5537 return false; 5538 5539 bfr.popFront(0); 5540 if(bfr.sourceClosed) 5541 return false; 5542 goto top; 5543 } 5544 5545 5546 bool isDataPending(Duration timeout = 0.seconds) { 5547 Socket socket = cgi.idlol.source; 5548 5549 auto check = new SocketSet(); 5550 check.add(socket); 5551 5552 auto got = Socket.select(check, null, null, timeout); 5553 if(got > 0) 5554 return true; 5555 return false; 5556 } 5557 5558 // note: this blocks 5559 WebSocketFrame recv() { 5560 return waitForNextMessage(); 5561 } 5562 5563 5564 5565 5566 private void llclose() { 5567 cgi.close(); 5568 } 5569 5570 private void llsend(ubyte[] data) { 5571 cgi.write(data); 5572 cgi.flush(); 5573 } 5574 5575 void unregisterActiveSocket(WebSocket) {} 5576 5577 /* copy/paste section { */ 5578 5579 private int readyState_; 5580 private ubyte[] receiveBuffer; 5581 private size_t receiveBufferUsedLength; 5582 5583 private Config config; 5584 5585 enum CONNECTING = 0; /// Socket has been created. The connection is not yet open. 5586 enum OPEN = 1; /// The connection is open and ready to communicate. 5587 enum CLOSING = 2; /// The connection is in the process of closing. 5588 enum CLOSED = 3; /// The connection is closed or couldn't be opened. 5589 5590 /++ 5591 5592 +/ 5593 /// Group: foundational 5594 static struct Config { 5595 /++ 5596 These control the size of the receive buffer. 5597 5598 It starts at the initial size, will temporarily 5599 balloon up to the maximum size, and will reuse 5600 a buffer up to the likely size. 5601 5602 Anything larger than the maximum size will cause 5603 the connection to be aborted and an exception thrown. 5604 This is to protect you against a peer trying to 5605 exhaust your memory, while keeping the user-level 5606 processing simple. 5607 +/ 5608 size_t initialReceiveBufferSize = 4096; 5609 size_t likelyReceiveBufferSize = 4096; /// ditto 5610 size_t maximumReceiveBufferSize = 10 * 1024 * 1024; /// ditto 5611 5612 /++ 5613 Maximum combined size of a message. 5614 +/ 5615 size_t maximumMessageSize = 10 * 1024 * 1024; 5616 5617 string[string] cookies; /// Cookies to send with the initial request. cookies[name] = value; 5618 string origin; /// Origin URL to send with the handshake, if desired. 5619 string protocol; /// the protocol header, if desired. 5620 5621 int pingFrequency = 5000; /// Amount of time (in msecs) of idleness after which to send an automatic ping 5622 } 5623 5624 /++ 5625 Returns one of [CONNECTING], [OPEN], [CLOSING], or [CLOSED]. 5626 +/ 5627 int readyState() { 5628 return readyState_; 5629 } 5630 5631 /++ 5632 Closes the connection, sending a graceful teardown message to the other side. 5633 +/ 5634 /// Group: foundational 5635 void close(int code = 0, string reason = null) 5636 //in (reason.length < 123) 5637 in { assert(reason.length < 123); } do 5638 { 5639 if(readyState_ != OPEN) 5640 return; // it cool, we done 5641 WebSocketFrame wss; 5642 wss.fin = true; 5643 wss.opcode = WebSocketOpcode.close; 5644 wss.data = cast(ubyte[]) reason.dup; 5645 wss.send(&llsend); 5646 5647 readyState_ = CLOSING; 5648 5649 llclose(); 5650 } 5651 5652 /++ 5653 Sends a ping message to the server. This is done automatically by the library if you set a non-zero [Config.pingFrequency], but you can also send extra pings explicitly as well with this function. 5654 +/ 5655 /// Group: foundational 5656 void ping() { 5657 WebSocketFrame wss; 5658 wss.fin = true; 5659 wss.opcode = WebSocketOpcode.ping; 5660 wss.send(&llsend); 5661 } 5662 5663 // automatically handled.... 5664 void pong() { 5665 WebSocketFrame wss; 5666 wss.fin = true; 5667 wss.opcode = WebSocketOpcode.pong; 5668 wss.send(&llsend); 5669 } 5670 5671 /++ 5672 Sends a text message through the websocket. 5673 +/ 5674 /// Group: foundational 5675 void send(in char[] textData) { 5676 WebSocketFrame wss; 5677 wss.fin = true; 5678 wss.opcode = WebSocketOpcode.text; 5679 wss.data = cast(ubyte[]) textData.dup; 5680 wss.send(&llsend); 5681 } 5682 5683 /++ 5684 Sends a binary message through the websocket. 5685 +/ 5686 /// Group: foundational 5687 void send(in ubyte[] binaryData) { 5688 WebSocketFrame wss; 5689 wss.fin = true; 5690 wss.opcode = WebSocketOpcode.binary; 5691 wss.data = cast(ubyte[]) binaryData.dup; 5692 wss.send(&llsend); 5693 } 5694 5695 /++ 5696 Waits for and returns the next complete message on the socket. 5697 5698 Note that the onmessage function is still called, right before 5699 this returns. 5700 +/ 5701 /// Group: blocking_api 5702 public WebSocketFrame waitForNextMessage() { 5703 do { 5704 auto m = processOnce(); 5705 if(m.populated) 5706 return m; 5707 } while(lowLevelReceive()); 5708 5709 throw new ConnectionClosedException("Websocket receive timed out"); 5710 //return WebSocketFrame.init; // FIXME? maybe. 5711 } 5712 5713 /++ 5714 Tells if [waitForNextMessage] would block. 5715 +/ 5716 /// Group: blocking_api 5717 public bool waitForNextMessageWouldBlock() { 5718 checkAgain: 5719 if(isMessageBuffered()) 5720 return false; 5721 if(!isDataPending()) 5722 return true; 5723 while(isDataPending()) 5724 lowLevelReceive(); 5725 goto checkAgain; 5726 } 5727 5728 /++ 5729 Is there a message in the buffer already? 5730 If `true`, [waitForNextMessage] is guaranteed to return immediately. 5731 If `false`, check [isDataPending] as the next step. 5732 +/ 5733 /// Group: blocking_api 5734 public bool isMessageBuffered() { 5735 ubyte[] d = receiveBuffer[0 .. receiveBufferUsedLength]; 5736 auto s = d; 5737 if(d.length) { 5738 auto orig = d; 5739 auto m = WebSocketFrame.read(d); 5740 // that's how it indicates that it needs more data 5741 if(d !is orig) 5742 return true; 5743 } 5744 5745 return false; 5746 } 5747 5748 private ubyte continuingType; 5749 private ubyte[] continuingData; 5750 //private size_t continuingDataLength; 5751 5752 private WebSocketFrame processOnce() { 5753 ubyte[] d = receiveBuffer[0 .. receiveBufferUsedLength]; 5754 auto s = d; 5755 // FIXME: handle continuation frames more efficiently. it should really just reuse the receive buffer. 5756 WebSocketFrame m; 5757 if(d.length) { 5758 auto orig = d; 5759 m = WebSocketFrame.read(d); 5760 // that's how it indicates that it needs more data 5761 if(d is orig) 5762 return WebSocketFrame.init; 5763 m.unmaskInPlace(); 5764 switch(m.opcode) { 5765 case WebSocketOpcode.continuation: 5766 if(continuingData.length + m.data.length > config.maximumMessageSize) 5767 throw new Exception("message size exceeded"); 5768 5769 continuingData ~= m.data; 5770 if(m.fin) { 5771 if(ontextmessage) 5772 ontextmessage(cast(char[]) continuingData); 5773 if(onbinarymessage) 5774 onbinarymessage(continuingData); 5775 5776 continuingData = null; 5777 } 5778 break; 5779 case WebSocketOpcode.text: 5780 if(m.fin) { 5781 if(ontextmessage) 5782 ontextmessage(m.textData); 5783 } else { 5784 continuingType = m.opcode; 5785 //continuingDataLength = 0; 5786 continuingData = null; 5787 continuingData ~= m.data; 5788 } 5789 break; 5790 case WebSocketOpcode.binary: 5791 if(m.fin) { 5792 if(onbinarymessage) 5793 onbinarymessage(m.data); 5794 } else { 5795 continuingType = m.opcode; 5796 //continuingDataLength = 0; 5797 continuingData = null; 5798 continuingData ~= m.data; 5799 } 5800 break; 5801 case WebSocketOpcode.close: 5802 readyState_ = CLOSED; 5803 if(onclose) 5804 onclose(); 5805 5806 unregisterActiveSocket(this); 5807 break; 5808 case WebSocketOpcode.ping: 5809 pong(); 5810 break; 5811 case WebSocketOpcode.pong: 5812 // just really references it is still alive, nbd. 5813 break; 5814 default: // ignore though i could and perhaps should throw too 5815 } 5816 } 5817 5818 // the recv thing can be invalidated so gotta copy it over ugh 5819 if(d.length) { 5820 m.data = m.data.dup(); 5821 } 5822 5823 import core.stdc.string; 5824 memmove(receiveBuffer.ptr, d.ptr, d.length); 5825 receiveBufferUsedLength = d.length; 5826 5827 return m; 5828 } 5829 5830 private void autoprocess() { 5831 // FIXME 5832 do { 5833 processOnce(); 5834 } while(lowLevelReceive()); 5835 } 5836 5837 5838 void delegate() onclose; /// 5839 void delegate() onerror; /// 5840 void delegate(in char[]) ontextmessage; /// 5841 void delegate(in ubyte[]) onbinarymessage; /// 5842 void delegate() onopen; /// 5843 5844 /++ 5845 5846 +/ 5847 /// Group: browser_api 5848 void onmessage(void delegate(in char[]) dg) { 5849 ontextmessage = dg; 5850 } 5851 5852 /// ditto 5853 void onmessage(void delegate(in ubyte[]) dg) { 5854 onbinarymessage = dg; 5855 } 5856 5857 /* } end copy/paste */ 5858 5859 5860 } 5861 5862 bool websocketRequested(Cgi cgi) { 5863 return 5864 "sec-websocket-key" in cgi.requestHeaders 5865 && 5866 "connection" in cgi.requestHeaders && 5867 cgi.requestHeaders["connection"].asLowerCase().canFind("upgrade") 5868 && 5869 "upgrade" in cgi.requestHeaders && 5870 cgi.requestHeaders["upgrade"].asLowerCase().equal("websocket") 5871 ; 5872 } 5873 5874 WebSocket acceptWebsocket(Cgi cgi) { 5875 assert(!cgi.closed); 5876 assert(!cgi.outputtedResponseData); 5877 cgi.setResponseStatus("101 Switching Protocols"); 5878 cgi.header("Upgrade: WebSocket"); 5879 cgi.header("Connection: upgrade"); 5880 5881 string key = cgi.requestHeaders["sec-websocket-key"]; 5882 key ~= "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"; // the defined guid from the websocket spec 5883 5884 import std.digest.sha; 5885 auto hash = sha1Of(key); 5886 auto accept = Base64.encode(hash); 5887 5888 cgi.header(("Sec-WebSocket-Accept: " ~ accept).idup); 5889 5890 cgi.websocketMode = true; 5891 cgi.write(""); 5892 5893 cgi.flush(); 5894 5895 return new WebSocket(cgi); 5896 } 5897 5898 // FIXME get websocket to work on other modes, not just embedded_httpd 5899 5900 /* copy/paste in http2.d { */ 5901 enum WebSocketOpcode : ubyte { 5902 continuation = 0, 5903 text = 1, 5904 binary = 2, 5905 // 3, 4, 5, 6, 7 RESERVED 5906 close = 8, 5907 ping = 9, 5908 pong = 10, 5909 // 11,12,13,14,15 RESERVED 5910 } 5911 5912 public struct WebSocketFrame { 5913 private bool populated; 5914 bool fin; 5915 bool rsv1; 5916 bool rsv2; 5917 bool rsv3; 5918 WebSocketOpcode opcode; // 4 bits 5919 bool masked; 5920 ubyte lengthIndicator; // don't set this when building one to send 5921 ulong realLength; // don't use when sending 5922 ubyte[4] maskingKey; // don't set this when sending 5923 ubyte[] data; 5924 5925 static WebSocketFrame simpleMessage(WebSocketOpcode opcode, void[] data) { 5926 WebSocketFrame msg; 5927 msg.fin = true; 5928 msg.opcode = opcode; 5929 msg.data = cast(ubyte[]) data.dup; 5930 5931 return msg; 5932 } 5933 5934 private void send(scope void delegate(ubyte[]) llsend) { 5935 ubyte[64] headerScratch; 5936 int headerScratchPos = 0; 5937 5938 realLength = data.length; 5939 5940 { 5941 ubyte b1; 5942 b1 |= cast(ubyte) opcode; 5943 b1 |= rsv3 ? (1 << 4) : 0; 5944 b1 |= rsv2 ? (1 << 5) : 0; 5945 b1 |= rsv1 ? (1 << 6) : 0; 5946 b1 |= fin ? (1 << 7) : 0; 5947 5948 headerScratch[0] = b1; 5949 headerScratchPos++; 5950 } 5951 5952 { 5953 headerScratchPos++; // we'll set header[1] at the end of this 5954 auto rlc = realLength; 5955 ubyte b2; 5956 b2 |= masked ? (1 << 7) : 0; 5957 5958 assert(headerScratchPos == 2); 5959 5960 if(realLength > 65535) { 5961 // use 64 bit length 5962 b2 |= 0x7f; 5963 5964 // FIXME: double check endinaness 5965 foreach(i; 0 .. 8) { 5966 headerScratch[2 + 7 - i] = rlc & 0x0ff; 5967 rlc >>>= 8; 5968 } 5969 5970 headerScratchPos += 8; 5971 } else if(realLength > 125) { 5972 // use 16 bit length 5973 b2 |= 0x7e; 5974 5975 // FIXME: double check endinaness 5976 foreach(i; 0 .. 2) { 5977 headerScratch[2 + 1 - i] = rlc & 0x0ff; 5978 rlc >>>= 8; 5979 } 5980 5981 headerScratchPos += 2; 5982 } else { 5983 // use 7 bit length 5984 b2 |= realLength & 0b_0111_1111; 5985 } 5986 5987 headerScratch[1] = b2; 5988 } 5989 5990 //assert(!masked, "masking key not properly implemented"); 5991 if(masked) { 5992 // FIXME: randomize this 5993 headerScratch[headerScratchPos .. headerScratchPos + 4] = maskingKey[]; 5994 headerScratchPos += 4; 5995 5996 // we'll just mask it in place... 5997 int keyIdx = 0; 5998 foreach(i; 0 .. data.length) { 5999 data[i] = data[i] ^ maskingKey[keyIdx]; 6000 if(keyIdx == 3) 6001 keyIdx = 0; 6002 else 6003 keyIdx++; 6004 } 6005 } 6006 6007 //writeln("SENDING ", headerScratch[0 .. headerScratchPos], data); 6008 llsend(headerScratch[0 .. headerScratchPos]); 6009 llsend(data); 6010 } 6011 6012 static WebSocketFrame read(ref ubyte[] d) { 6013 WebSocketFrame msg; 6014 6015 auto orig = d; 6016 6017 WebSocketFrame needsMoreData() { 6018 d = orig; 6019 return WebSocketFrame.init; 6020 } 6021 6022 if(d.length < 2) 6023 return needsMoreData(); 6024 6025 ubyte b = d[0]; 6026 6027 msg.populated = true; 6028 6029 msg.opcode = cast(WebSocketOpcode) (b & 0x0f); 6030 b >>= 4; 6031 msg.rsv3 = b & 0x01; 6032 b >>= 1; 6033 msg.rsv2 = b & 0x01; 6034 b >>= 1; 6035 msg.rsv1 = b & 0x01; 6036 b >>= 1; 6037 msg.fin = b & 0x01; 6038 6039 b = d[1]; 6040 msg.masked = (b & 0b1000_0000) ? true : false; 6041 msg.lengthIndicator = b & 0b0111_1111; 6042 6043 d = d[2 .. $]; 6044 6045 if(msg.lengthIndicator == 0x7e) { 6046 // 16 bit length 6047 msg.realLength = 0; 6048 6049 if(d.length < 2) return needsMoreData(); 6050 6051 foreach(i; 0 .. 2) { 6052 msg.realLength |= d[0] << ((1-i) * 8); 6053 d = d[1 .. $]; 6054 } 6055 } else if(msg.lengthIndicator == 0x7f) { 6056 // 64 bit length 6057 msg.realLength = 0; 6058 6059 if(d.length < 8) return needsMoreData(); 6060 6061 foreach(i; 0 .. 8) { 6062 msg.realLength |= ulong(d[0]) << ((7-i) * 8); 6063 d = d[1 .. $]; 6064 } 6065 } else { 6066 // 7 bit length 6067 msg.realLength = msg.lengthIndicator; 6068 } 6069 6070 if(msg.masked) { 6071 6072 if(d.length < 4) return needsMoreData(); 6073 6074 msg.maskingKey = d[0 .. 4]; 6075 d = d[4 .. $]; 6076 } 6077 6078 if(msg.realLength > d.length) { 6079 return needsMoreData(); 6080 } 6081 6082 msg.data = d[0 .. cast(size_t) msg.realLength]; 6083 d = d[cast(size_t) msg.realLength .. $]; 6084 6085 return msg; 6086 } 6087 6088 void unmaskInPlace() { 6089 if(this.masked) { 6090 int keyIdx = 0; 6091 foreach(i; 0 .. this.data.length) { 6092 this.data[i] = this.data[i] ^ this.maskingKey[keyIdx]; 6093 if(keyIdx == 3) 6094 keyIdx = 0; 6095 else 6096 keyIdx++; 6097 } 6098 } 6099 } 6100 6101 char[] textData() { 6102 return cast(char[]) data; 6103 } 6104 } 6105 /* } */ 6106 } 6107 6108 6109 version(Windows) 6110 { 6111 version(CRuntime_DigitalMars) 6112 { 6113 extern(C) int setmode(int, int) nothrow @nogc; 6114 } 6115 else version(CRuntime_Microsoft) 6116 { 6117 extern(C) int _setmode(int, int) nothrow @nogc; 6118 alias setmode = _setmode; 6119 } 6120 else static assert(0); 6121 } 6122 6123 version(Posix) { 6124 version(CRuntime_Musl) {} else { 6125 import core.sys.posix.unistd; 6126 private extern(C) int posix_spawn(pid_t*, const char*, void*, void*, const char**, const char**); 6127 } 6128 } 6129 6130 6131 // FIXME: these aren't quite public yet. 6132 //private: 6133 6134 // template for laziness 6135 void startAddonServer()(string arg) { 6136 version(OSX) { 6137 assert(0, "Not implemented"); 6138 } else version(linux) { 6139 import core.sys.posix.unistd; 6140 pid_t pid; 6141 const(char)*[16] args; 6142 args[0] = "ARSD_CGI_ADDON_SERVER"; 6143 args[1] = arg.ptr; 6144 posix_spawn(&pid, "/proc/self/exe", 6145 null, 6146 null, 6147 args.ptr, 6148 null // env 6149 ); 6150 } else version(Windows) { 6151 wchar[2048] filename; 6152 auto len = GetModuleFileNameW(null, filename.ptr, cast(DWORD) filename.length); 6153 if(len == 0 || len == filename.length) 6154 throw new Exception("could not get process name to start helper server"); 6155 6156 STARTUPINFOW startupInfo; 6157 startupInfo.cb = cast(DWORD) startupInfo.sizeof; 6158 PROCESS_INFORMATION processInfo; 6159 6160 import std.utf; 6161 6162 // I *MIGHT* need to run it as a new job or a service... 6163 auto ret = CreateProcessW( 6164 filename.ptr, 6165 toUTF16z(arg), 6166 null, // process attributes 6167 null, // thread attributes 6168 false, // inherit handles 6169 0, // creation flags 6170 null, // environment 6171 null, // working directory 6172 &startupInfo, 6173 &processInfo 6174 ); 6175 6176 if(!ret) 6177 throw new Exception("create process failed"); 6178 6179 // when done with those, if we set them 6180 /* 6181 CloseHandle(hStdInput); 6182 CloseHandle(hStdOutput); 6183 CloseHandle(hStdError); 6184 */ 6185 6186 } else static assert(0, "Websocket server not implemented on this system yet (email me, i can prolly do it if you need it)"); 6187 } 6188 6189 // template for laziness 6190 /* 6191 The websocket server is a single-process, single-thread, event 6192 I/O thing. It is passed websockets from other CGI processes 6193 and is then responsible for handling their messages and responses. 6194 Note that the CGI process is responsible for websocket setup, 6195 including authentication, etc. 6196 6197 It also gets data sent to it by other processes and is responsible 6198 for distributing that, as necessary. 6199 */ 6200 void runWebsocketServer()() { 6201 assert(0, "not implemented"); 6202 } 6203 6204 void sendToWebsocketServer(WebSocket ws, string group) { 6205 assert(0, "not implemented"); 6206 } 6207 6208 void sendToWebsocketServer(string content, string group) { 6209 assert(0, "not implemented"); 6210 } 6211 6212 6213 void runEventServer()() { 6214 runAddonServer("/tmp/arsd_cgi_event_server", new EventSourceServerImplementation()); 6215 } 6216 6217 void runTimerServer()() { 6218 runAddonServer("/tmp/arsd_scheduled_job_server", new ScheduledJobServerImplementation()); 6219 } 6220 6221 version(Posix) { 6222 alias LocalServerConnectionHandle = int; 6223 alias CgiConnectionHandle = int; 6224 alias SocketConnectionHandle = int; 6225 6226 enum INVALID_CGI_CONNECTION_HANDLE = -1; 6227 } else version(Windows) { 6228 alias LocalServerConnectionHandle = HANDLE; 6229 version(embedded_httpd_threads) { 6230 alias CgiConnectionHandle = SOCKET; 6231 enum INVALID_CGI_CONNECTION_HANDLE = INVALID_SOCKET; 6232 } else version(fastcgi) { 6233 alias CgiConnectionHandle = void*; // Doesn't actually work! But I don't want compile to fail pointlessly at this point. 6234 enum INVALID_CGI_CONNECTION_HANDLE = null; 6235 } else version(scgi) { 6236 alias CgiConnectionHandle = SOCKET; 6237 enum INVALID_CGI_CONNECTION_HANDLE = INVALID_SOCKET; 6238 } else { /* version(plain_cgi) */ 6239 alias CgiConnectionHandle = HANDLE; 6240 enum INVALID_CGI_CONNECTION_HANDLE = null; 6241 } 6242 alias SocketConnectionHandle = SOCKET; 6243 } 6244 6245 version(with_addon_servers_connections) 6246 LocalServerConnectionHandle openLocalServerConnection()(string name, string arg) { 6247 version(Posix) { 6248 import core.sys.posix.unistd; 6249 import core.sys.posix.sys.un; 6250 6251 int sock = socket(AF_UNIX, SOCK_STREAM, 0); 6252 if(sock == -1) 6253 throw new Exception("socket " ~ to!string(errno)); 6254 6255 scope(failure) 6256 close(sock); 6257 6258 cloexec(sock); 6259 6260 // add-on server processes are assumed to be local, and thus will 6261 // use unix domain sockets. Besides, I want to pass sockets to them, 6262 // so it basically must be local (except for the session server, but meh). 6263 sockaddr_un addr; 6264 addr.sun_family = AF_UNIX; 6265 version(linux) { 6266 // on linux, we will use the abstract namespace 6267 addr.sun_path[0] = 0; 6268 addr.sun_path[1 .. name.length + 1] = cast(typeof(addr.sun_path[])) name[]; 6269 } else { 6270 // but otherwise, just use a file cuz we must. 6271 addr.sun_path[0 .. name.length] = cast(typeof(addr.sun_path[])) name[]; 6272 } 6273 6274 bool alreadyTried; 6275 6276 try_again: 6277 6278 if(connect(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) { 6279 if(!alreadyTried && errno == ECONNREFUSED) { 6280 // try auto-spawning the server, then attempt connection again 6281 startAddonServer(arg); 6282 import core.thread; 6283 Thread.sleep(50.msecs); 6284 alreadyTried = true; 6285 goto try_again; 6286 } else 6287 throw new Exception("connect " ~ to!string(errno)); 6288 } 6289 6290 return sock; 6291 } else version(Windows) { 6292 return null; // FIXME 6293 } 6294 } 6295 6296 version(with_addon_servers_connections) 6297 void closeLocalServerConnection(LocalServerConnectionHandle handle) { 6298 version(Posix) { 6299 import core.sys.posix.unistd; 6300 close(handle); 6301 } else version(Windows) 6302 CloseHandle(handle); 6303 } 6304 6305 void runSessionServer()() { 6306 runAddonServer("/tmp/arsd_session_server", new BasicDataServerImplementation()); 6307 } 6308 6309 version(Posix) 6310 private void makeNonBlocking(int fd) { 6311 import core.sys.posix.fcntl; 6312 auto flags = fcntl(fd, F_GETFL, 0); 6313 if(flags == -1) 6314 throw new Exception("fcntl get"); 6315 flags |= O_NONBLOCK; 6316 auto s = fcntl(fd, F_SETFL, flags); 6317 if(s == -1) 6318 throw new Exception("fcntl set"); 6319 } 6320 6321 import core.stdc.errno; 6322 6323 struct IoOp { 6324 @disable this(); 6325 @disable this(this); 6326 6327 /* 6328 So we want to be able to eventually handle generic sockets too. 6329 */ 6330 6331 enum Read = 1; 6332 enum Write = 2; 6333 enum Accept = 3; 6334 enum ReadSocketHandle = 4; 6335 6336 // Your handler may be called in a different thread than the one that initiated the IO request! 6337 // It is also possible to have multiple io requests being called simultaneously. Use proper thread safety caution. 6338 private bool delegate(IoOp*, int) handler; // returns true if you are done and want it to be closed 6339 private void delegate(IoOp*) closeHandler; 6340 private void delegate(IoOp*) completeHandler; 6341 private int internalFd; 6342 private int operation; 6343 private int bufferLengthAllocated; 6344 private int bufferLengthUsed; 6345 private ubyte[1] internalBuffer; // it can be overallocated! 6346 6347 ubyte[] allocatedBuffer() return { 6348 return internalBuffer.ptr[0 .. bufferLengthAllocated]; 6349 } 6350 6351 ubyte[] usedBuffer() return { 6352 return allocatedBuffer[0 .. bufferLengthUsed]; 6353 } 6354 6355 void reset() { 6356 bufferLengthUsed = 0; 6357 } 6358 6359 int fd() { 6360 return internalFd; 6361 } 6362 } 6363 6364 IoOp* allocateIoOp(int fd, int operation, int bufferSize, bool delegate(IoOp*, int) handler) { 6365 import core.stdc.stdlib; 6366 6367 auto ptr = calloc(IoOp.sizeof + bufferSize, 1); 6368 if(ptr is null) 6369 assert(0); // out of memory! 6370 6371 auto op = cast(IoOp*) ptr; 6372 6373 op.handler = handler; 6374 op.internalFd = fd; 6375 op.operation = operation; 6376 op.bufferLengthAllocated = bufferSize; 6377 op.bufferLengthUsed = 0; 6378 6379 import core.memory; 6380 6381 GC.addRoot(ptr); 6382 6383 return op; 6384 } 6385 6386 void freeIoOp(ref IoOp* ptr) { 6387 6388 import core.memory; 6389 GC.removeRoot(ptr); 6390 6391 import core.stdc.stdlib; 6392 free(ptr); 6393 ptr = null; 6394 } 6395 6396 version(Posix) 6397 version(with_addon_servers_connections) 6398 void nonBlockingWrite(EventIoServer eis, int connection, const void[] data) { 6399 import core.sys.posix.unistd; 6400 6401 auto ret = write(connection, data.ptr, data.length); 6402 if(ret != data.length) { 6403 if(ret == 0 || errno == EPIPE) { 6404 // the file is closed, remove it 6405 eis.fileClosed(connection); 6406 } else 6407 throw new Exception("alas " ~ to!string(ret) ~ " " ~ to!string(errno)); // FIXME 6408 } 6409 } 6410 version(Windows) 6411 version(with_addon_servers_connections) 6412 void nonBlockingWrite(EventIoServer eis, int connection, const void[] data) { 6413 // FIXME 6414 } 6415 6416 bool isInvalidHandle(CgiConnectionHandle h) { 6417 return h == INVALID_CGI_CONNECTION_HANDLE; 6418 } 6419 6420 /+ 6421 https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsarecv 6422 https://support.microsoft.com/en-gb/help/181611/socket-overlapped-i-o-versus-blocking-nonblocking-mode 6423 https://stackoverflow.com/questions/18018489/should-i-use-iocps-or-overlapped-wsasend-receive 6424 https://docs.microsoft.com/en-us/windows/desktop/fileio/i-o-completion-ports 6425 https://docs.microsoft.com/en-us/windows/desktop/fileio/createiocompletionport 6426 https://docs.microsoft.com/en-us/windows/desktop/api/mswsock/nf-mswsock-acceptex 6427 https://docs.microsoft.com/en-us/windows/desktop/Sync/waitable-timer-objects 6428 https://docs.microsoft.com/en-us/windows/desktop/api/synchapi/nf-synchapi-setwaitabletimer 6429 https://docs.microsoft.com/en-us/windows/desktop/Sync/using-a-waitable-timer-with-an-asynchronous-procedure-call 6430 https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsagetoverlappedresult 6431 6432 +/ 6433 6434 /++ 6435 You can customize your server by subclassing the appropriate server. Then, register your 6436 subclass at compile time with the [registerEventIoServer] template, or implement your own 6437 main function and call it yourself. 6438 6439 $(TIP If you make your subclass a `final class`, there is a slight performance improvement.) 6440 +/ 6441 version(with_addon_servers_connections) 6442 interface EventIoServer { 6443 bool handleLocalConnectionData(IoOp* op, int receivedFd); 6444 void handleLocalConnectionClose(IoOp* op); 6445 void handleLocalConnectionComplete(IoOp* op); 6446 void wait_timeout(); 6447 void fileClosed(int fd); 6448 6449 void epoll_fd(int fd); 6450 } 6451 6452 // the sink should buffer it 6453 private void serialize(T)(scope void delegate(ubyte[]) sink, T t) { 6454 static if(is(T == struct)) { 6455 foreach(member; __traits(allMembers, T)) 6456 serialize(sink, __traits(getMember, t, member)); 6457 } else static if(is(T : int)) { 6458 // no need to think of endianness just because this is only used 6459 // for local, same-machine stuff anyway. thanks private lol 6460 sink((cast(ubyte*) &t)[0 .. t.sizeof]); 6461 } else static if(is(T == string) || is(T : const(ubyte)[])) { 6462 // these are common enough to optimize 6463 int len = cast(int) t.length; // want length consistent size tho, in case 32 bit program sends to 64 bit server, etc. 6464 sink((cast(ubyte*) &len)[0 .. int.sizeof]); 6465 sink(cast(ubyte[]) t[]); 6466 } else static if(is(T : A[], A)) { 6467 // generic array is less optimal but still prolly ok 6468 int len = cast(int) t.length; 6469 sink((cast(ubyte*) &len)[0 .. int.sizeof]); 6470 foreach(item; t) 6471 serialize(sink, item); 6472 } else static assert(0, T.stringof); 6473 } 6474 6475 // all may be stack buffers, so use cautio 6476 private void deserialize(T)(scope ubyte[] delegate(int sz) get, scope void delegate(T) dg) { 6477 static if(is(T == struct)) { 6478 T t; 6479 foreach(member; __traits(allMembers, T)) 6480 deserialize!(typeof(__traits(getMember, T, member)))(get, (mbr) { __traits(getMember, t, member) = mbr; }); 6481 dg(t); 6482 } else static if(is(T : int)) { 6483 // no need to think of endianness just because this is only used 6484 // for local, same-machine stuff anyway. thanks private lol 6485 T t; 6486 auto data = get(t.sizeof); 6487 t = (cast(T[]) data)[0]; 6488 dg(t); 6489 } else static if(is(T == string) || is(T : const(ubyte)[])) { 6490 // these are common enough to optimize 6491 int len; 6492 auto data = get(len.sizeof); 6493 len = (cast(int[]) data)[0]; 6494 6495 /* 6496 typeof(T[0])[2000] stackBuffer; 6497 T buffer; 6498 6499 if(len < stackBuffer.length) 6500 buffer = stackBuffer[0 .. len]; 6501 else 6502 buffer = new T(len); 6503 6504 data = get(len * typeof(T[0]).sizeof); 6505 */ 6506 6507 T t = cast(T) get(len * cast(int) typeof(T.init[0]).sizeof); 6508 6509 dg(t); 6510 } else static if(is(T == E[], E)) { 6511 T t; 6512 int len; 6513 auto data = get(len.sizeof); 6514 len = (cast(int[]) data)[0]; 6515 t.length = len; 6516 foreach(ref e; t) { 6517 deserialize!E(get, (ele) { e = ele; }); 6518 } 6519 dg(t); 6520 } else static assert(0, T.stringof); 6521 } 6522 6523 unittest { 6524 serialize((ubyte[] b) { 6525 deserialize!int( sz => b[0 .. sz], (t) { assert(t == 1); }); 6526 }, 1); 6527 serialize((ubyte[] b) { 6528 deserialize!int( sz => b[0 .. sz], (t) { assert(t == 56674); }); 6529 }, 56674); 6530 ubyte[1000] buffer; 6531 int bufferPoint; 6532 void add(ubyte[] b) { 6533 buffer[bufferPoint .. bufferPoint + b.length] = b[]; 6534 bufferPoint += b.length; 6535 } 6536 ubyte[] get(int sz) { 6537 auto b = buffer[bufferPoint .. bufferPoint + sz]; 6538 bufferPoint += sz; 6539 return b; 6540 } 6541 serialize(&add, "test here"); 6542 bufferPoint = 0; 6543 deserialize!string(&get, (t) { assert(t == "test here"); }); 6544 bufferPoint = 0; 6545 6546 struct Foo { 6547 int a; 6548 ubyte c; 6549 string d; 6550 } 6551 serialize(&add, Foo(403, 37, "amazing")); 6552 bufferPoint = 0; 6553 deserialize!Foo(&get, (t) { 6554 assert(t.a == 403); 6555 assert(t.c == 37); 6556 assert(t.d == "amazing"); 6557 }); 6558 bufferPoint = 0; 6559 } 6560 6561 /* 6562 Here's the way the RPC interface works: 6563 6564 You define the interface that lists the functions you can call on the remote process. 6565 The interface may also have static methods for convenience. These forward to a singleton 6566 instance of an auto-generated class, which actually sends the args over the pipe. 6567 6568 An impl class actually implements it. A receiving server deserializes down the pipe and 6569 calls methods on the class. 6570 6571 I went with the interface to get some nice compiler checking and documentation stuff. 6572 6573 I could have skipped the interface and just implemented it all from the server class definition 6574 itself, but then the usage may call the method instead of rpcing it; I just like having the user 6575 interface and the implementation separate so you aren't tempted to `new impl` to call the methods. 6576 6577 6578 I fiddled with newlines in the mixin string to ensure the assert line numbers matched up to the source code line number. Idk why dmd didn't do this automatically, but it was important to me. 6579 6580 Realistically though the bodies would just be 6581 connection.call(this.mangleof, args...) sooooo. 6582 6583 FIXME: overloads aren't supported 6584 */ 6585 6586 /// Base for storing sessions in an array. Exists primarily for internal purposes and you should generally not use this. 6587 interface SessionObject {} 6588 6589 private immutable void delegate(string[])[string] scheduledJobHandlers; 6590 private immutable void delegate(string[])[string] websocketServers; 6591 6592 version(with_breaking_cgi_features) 6593 mixin(q{ 6594 6595 mixin template ImplementRpcClientInterface(T, string serverPath, string cmdArg) { 6596 static import std.traits; 6597 6598 // derivedMembers on an interface seems to give exactly what I want: the virtual functions we need to implement. so I am just going to use it directly without more filtering. 6599 static foreach(idx, member; __traits(derivedMembers, T)) { 6600 static if(__traits(isVirtualFunction, __traits(getMember, T, member))) 6601 mixin( q{ 6602 std.traits.ReturnType!(__traits(getMember, T, member)) 6603 } ~ member ~ q{(std.traits.Parameters!(__traits(getMember, T, member)) params) 6604 { 6605 SerializationBuffer buffer; 6606 auto i = cast(ushort) idx; 6607 serialize(&buffer.sink, i); 6608 serialize(&buffer.sink, __traits(getMember, T, member).mangleof); 6609 foreach(param; params) 6610 serialize(&buffer.sink, param); 6611 6612 auto sendable = buffer.sendable; 6613 6614 version(Posix) {{ 6615 auto ret = send(connectionHandle, sendable.ptr, sendable.length, 0); 6616 6617 if(ret == -1) { 6618 throw new Exception("send returned -1, errno: " ~ to!string(errno)); 6619 } else if(ret == 0) { 6620 throw new Exception("Connection to addon server lost"); 6621 } if(ret < sendable.length) 6622 throw new Exception("Send failed to send all"); 6623 assert(ret == sendable.length); 6624 }} // FIXME Windows impl 6625 6626 static if(!is(typeof(return) == void)) { 6627 // there is a return value; we need to wait for it too 6628 version(Posix) { 6629 ubyte[3000] revBuffer; 6630 auto ret = recv(connectionHandle, revBuffer.ptr, revBuffer.length, 0); 6631 auto got = revBuffer[0 .. ret]; 6632 6633 int dataLocation; 6634 ubyte[] grab(int sz) { 6635 auto d = got[dataLocation .. dataLocation + sz]; 6636 dataLocation += sz; 6637 return d; 6638 } 6639 6640 typeof(return) retu; 6641 deserialize!(typeof(return))(&grab, (a) { retu = a; }); 6642 return retu; 6643 } else { 6644 // FIXME Windows impl 6645 return typeof(return).init; 6646 } 6647 6648 } 6649 }}); 6650 } 6651 6652 private static typeof(this) singletonInstance; 6653 private LocalServerConnectionHandle connectionHandle; 6654 6655 static typeof(this) connection() { 6656 if(singletonInstance is null) { 6657 singletonInstance = new typeof(this)(); 6658 singletonInstance.connect(); 6659 } 6660 return singletonInstance; 6661 } 6662 6663 void connect() { 6664 connectionHandle = openLocalServerConnection(serverPath, cmdArg); 6665 } 6666 6667 void disconnect() { 6668 closeLocalServerConnection(connectionHandle); 6669 } 6670 } 6671 6672 void dispatchRpcServer(Interface, Class)(Class this_, ubyte[] data, int fd) if(is(Class : Interface)) { 6673 ushort calledIdx; 6674 string calledFunction; 6675 6676 int dataLocation; 6677 ubyte[] grab(int sz) { 6678 if(sz == 0) assert(0); 6679 auto d = data[dataLocation .. dataLocation + sz]; 6680 dataLocation += sz; 6681 return d; 6682 } 6683 6684 again: 6685 6686 deserialize!ushort(&grab, (a) { calledIdx = a; }); 6687 deserialize!string(&grab, (a) { calledFunction = a; }); 6688 6689 import std.traits; 6690 6691 sw: switch(calledIdx) { 6692 foreach(idx, memberName; __traits(derivedMembers, Interface)) 6693 static if(__traits(isVirtualFunction, __traits(getMember, Interface, memberName))) { 6694 case idx: 6695 assert(calledFunction == __traits(getMember, Interface, memberName).mangleof); 6696 6697 Parameters!(__traits(getMember, Interface, memberName)) params; 6698 foreach(ref param; params) 6699 deserialize!(typeof(param))(&grab, (a) { param = a; }); 6700 6701 static if(is(ReturnType!(__traits(getMember, Interface, memberName)) == void)) { 6702 __traits(getMember, this_, memberName)(params); 6703 } else { 6704 auto ret = __traits(getMember, this_, memberName)(params); 6705 SerializationBuffer buffer; 6706 serialize(&buffer.sink, ret); 6707 6708 auto sendable = buffer.sendable; 6709 6710 version(Posix) { 6711 auto r = send(fd, sendable.ptr, sendable.length, 0); 6712 if(r == -1) { 6713 throw new Exception("send returned -1, errno: " ~ to!string(errno)); 6714 } else if(r == 0) { 6715 throw new Exception("Connection to addon client lost"); 6716 } if(r < sendable.length) 6717 throw new Exception("Send failed to send all"); 6718 6719 } // FIXME Windows impl 6720 } 6721 break sw; 6722 } 6723 default: assert(0); 6724 } 6725 6726 if(dataLocation != data.length) 6727 goto again; 6728 } 6729 6730 6731 private struct SerializationBuffer { 6732 ubyte[2048] bufferBacking; 6733 int bufferLocation; 6734 void sink(scope ubyte[] data) { 6735 bufferBacking[bufferLocation .. bufferLocation + data.length] = data[]; 6736 bufferLocation += data.length; 6737 } 6738 6739 ubyte[] sendable() return { 6740 return bufferBacking[0 .. bufferLocation]; 6741 } 6742 } 6743 6744 /* 6745 FIXME: 6746 add a version command line arg 6747 version data in the library 6748 management gui as external program 6749 6750 at server with event_fd for each run 6751 use .mangleof in the at function name 6752 6753 i think the at server will have to: 6754 pipe args to the child 6755 collect child output for logging 6756 get child return value for logging 6757 6758 on windows timers work differently. idk how to best combine with the io stuff. 6759 6760 will have to have dump and restore too, so i can restart without losing stuff. 6761 */ 6762 6763 /++ 6764 A convenience object for talking to the [BasicDataServer] from a higher level. 6765 See: [Cgi.getSessionObject]. 6766 6767 You pass it a `Data` struct describing the data you want saved in the session. 6768 Then, this class will generate getter and setter properties that allow access 6769 to that data. 6770 6771 Note that each load and store will be done as-accessed; it doesn't front-load 6772 mutable data nor does it batch updates out of fear of read-modify-write race 6773 conditions. (In fact, right now it does this for everything, but in the future, 6774 I might batch load `immutable` members of the Data struct.) 6775 6776 At some point in the future, I might also let it do different backends, like 6777 a client-side cookie store too, but idk. 6778 6779 Note that the plain-old-data members of your `Data` struct are wrapped by this 6780 interface via a static foreach to make property functions. 6781 6782 See_Also: [MockSession] 6783 +/ 6784 interface Session(Data) : SessionObject { 6785 @property string sessionId() const; 6786 6787 /++ 6788 Starts a new session. Note that a session is also 6789 implicitly started as soon as you write data to it, 6790 so if you need to alter these parameters from their 6791 defaults, be sure to explicitly call this BEFORE doing 6792 any writes to session data. 6793 6794 Params: 6795 idleLifetime = How long, in seconds, the session 6796 should remain in memory when not being read from 6797 or written to. The default is one day. 6798 6799 NOT IMPLEMENTED 6800 6801 useExtendedLifetimeCookie = The session ID is always 6802 stored in a HTTP cookie, and by default, that cookie 6803 is discarded when the user closes their browser. 6804 6805 But if you set this to true, it will use a non-perishable 6806 cookie for the given idleLifetime. 6807 6808 NOT IMPLEMENTED 6809 +/ 6810 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false); 6811 6812 /++ 6813 Regenerates the session ID and updates the associated 6814 cookie. 6815 6816 This is also your chance to change immutable data 6817 (not yet implemented). 6818 +/ 6819 void regenerateId(); 6820 6821 /++ 6822 Terminates this session, deleting all saved data. 6823 +/ 6824 void terminate(); 6825 6826 /++ 6827 Plain-old-data members of your `Data` struct are wrapped here via 6828 the property getters and setters. 6829 6830 If the member is a non-string array, it returns a magical array proxy 6831 object which allows for atomic appends and replaces via overloaded operators. 6832 You can slice this to get a range representing a $(B const) view of the array. 6833 This is to protect you against read-modify-write race conditions. 6834 +/ 6835 static foreach(memberName; __traits(allMembers, Data)) 6836 static if(is(typeof(__traits(getMember, Data, memberName)))) 6837 mixin(q{ 6838 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout; 6839 @property typeof(__traits(getMember, Data, memberName)) } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value); 6840 }); 6841 6842 } 6843 6844 /++ 6845 An implementation of [Session] that works on real cgi connections utilizing the 6846 [BasicDataServer]. 6847 6848 As opposed to a [MockSession] which is made for testing purposes. 6849 6850 You will not construct one of these directly. See [Cgi.getSessionObject] instead. 6851 +/ 6852 class BasicDataServerSession(Data) : Session!Data { 6853 private Cgi cgi; 6854 private string sessionId_; 6855 6856 public @property string sessionId() const { 6857 return sessionId_; 6858 } 6859 6860 protected @property string sessionId(string s) { 6861 return this.sessionId_ = s; 6862 } 6863 6864 private this(Cgi cgi) { 6865 this.cgi = cgi; 6866 if(auto ptr = "sessionId" in cgi.cookies) 6867 sessionId = (*ptr).length ? *ptr : null; 6868 } 6869 6870 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false) { 6871 assert(sessionId is null); 6872 6873 // FIXME: what if there is a session ID cookie, but no corresponding session on the server? 6874 6875 import std.random, std.conv; 6876 sessionId = to!string(uniform(1, long.max)); 6877 6878 BasicDataServer.connection.createSession(sessionId, idleLifetime); 6879 setCookie(); 6880 } 6881 6882 protected void setCookie() { 6883 cgi.setCookie( 6884 "sessionId", sessionId, 6885 0 /* expiration */, 6886 "/" /* path */, 6887 null /* domain */, 6888 true /* http only */, 6889 cgi.https /* if the session is started on https, keep it there, otherwise, be flexible */); 6890 } 6891 6892 void regenerateId() { 6893 if(sessionId is null) { 6894 start(); 6895 return; 6896 } 6897 import std.random, std.conv; 6898 auto oldSessionId = sessionId; 6899 sessionId = to!string(uniform(1, long.max)); 6900 BasicDataServer.connection.renameSession(oldSessionId, sessionId); 6901 setCookie(); 6902 } 6903 6904 void terminate() { 6905 BasicDataServer.connection.destroySession(sessionId); 6906 sessionId = null; 6907 setCookie(); 6908 } 6909 6910 static foreach(memberName; __traits(allMembers, Data)) 6911 static if(is(typeof(__traits(getMember, Data, memberName)))) 6912 mixin(q{ 6913 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout { 6914 if(sessionId is null) 6915 return typeof(return).init; 6916 6917 import std.traits; 6918 auto v = BasicDataServer.connection.getSessionData(sessionId, fullyQualifiedName!Data ~ "." ~ memberName); 6919 if(v.length == 0) 6920 return typeof(return).init; 6921 import std.conv; 6922 // why this cast? to doesn't like being given an inout argument. so need to do it without that, then 6923 // we need to return it and that needed the cast. It should be fine since we basically respect constness.. 6924 // basically. Assuming the session is POD this should be fine. 6925 return cast(typeof(return)) to!(typeof(__traits(getMember, Data, memberName)))(v); 6926 } 6927 @property typeof(__traits(getMember, Data, memberName)) } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value) { 6928 if(sessionId is null) 6929 start(); 6930 import std.conv; 6931 import std.traits; 6932 BasicDataServer.connection.setSessionData(sessionId, fullyQualifiedName!Data ~ "." ~ memberName, to!string(value)); 6933 return value; 6934 } 6935 }); 6936 } 6937 6938 /++ 6939 A mock object that works like the real session, but doesn't actually interact with any actual database or http connection. 6940 Simply stores the data in its instance members. 6941 +/ 6942 class MockSession(Data) : Session!Data { 6943 pure { 6944 @property string sessionId() const { return "mock"; } 6945 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false) {} 6946 void regenerateId() {} 6947 void terminate() {} 6948 6949 private Data store_; 6950 6951 static foreach(memberName; __traits(allMembers, Data)) 6952 static if(is(typeof(__traits(getMember, Data, memberName)))) 6953 mixin(q{ 6954 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout { 6955 return __traits(getMember, store_, memberName); 6956 } 6957 @property typeof(__traits(getMember, Data, memberName)) } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value) { 6958 return __traits(getMember, store_, memberName) = value; 6959 } 6960 }); 6961 } 6962 } 6963 6964 /++ 6965 Direct interface to the basic data add-on server. You can 6966 typically use [Cgi.getSessionObject] as a more convenient interface. 6967 +/ 6968 version(with_addon_servers_connections) 6969 interface BasicDataServer { 6970 /// 6971 void createSession(string sessionId, int lifetime); 6972 /// 6973 void renewSession(string sessionId, int lifetime); 6974 /// 6975 void destroySession(string sessionId); 6976 /// 6977 void renameSession(string oldSessionId, string newSessionId); 6978 6979 /// 6980 void setSessionData(string sessionId, string dataKey, string dataValue); 6981 /// 6982 string getSessionData(string sessionId, string dataKey); 6983 6984 /// 6985 static BasicDataServerConnection connection() { 6986 return BasicDataServerConnection.connection(); 6987 } 6988 } 6989 6990 version(with_addon_servers_connections) 6991 class BasicDataServerConnection : BasicDataServer { 6992 mixin ImplementRpcClientInterface!(BasicDataServer, "/tmp/arsd_session_server", "--session-server"); 6993 } 6994 6995 version(with_addon_servers) 6996 final class BasicDataServerImplementation : BasicDataServer, EventIoServer { 6997 6998 void createSession(string sessionId, int lifetime) { 6999 sessions[sessionId.idup] = Session(lifetime); 7000 } 7001 void destroySession(string sessionId) { 7002 sessions.remove(sessionId); 7003 } 7004 void renewSession(string sessionId, int lifetime) { 7005 sessions[sessionId].lifetime = lifetime; 7006 } 7007 void renameSession(string oldSessionId, string newSessionId) { 7008 sessions[newSessionId.idup] = sessions[oldSessionId]; 7009 sessions.remove(oldSessionId); 7010 } 7011 void setSessionData(string sessionId, string dataKey, string dataValue) { 7012 if(sessionId !in sessions) 7013 createSession(sessionId, 3600); // FIXME? 7014 sessions[sessionId].values[dataKey.idup] = dataValue.idup; 7015 } 7016 string getSessionData(string sessionId, string dataKey) { 7017 if(auto session = sessionId in sessions) { 7018 if(auto data = dataKey in (*session).values) 7019 return *data; 7020 else 7021 return null; // no such data 7022 7023 } else { 7024 return null; // no session 7025 } 7026 } 7027 7028 7029 protected: 7030 7031 struct Session { 7032 int lifetime; 7033 7034 string[string] values; 7035 } 7036 7037 Session[string] sessions; 7038 7039 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 7040 auto data = op.usedBuffer; 7041 dispatchRpcServer!BasicDataServer(this, data, op.fd); 7042 return false; 7043 } 7044 7045 void handleLocalConnectionClose(IoOp* op) {} // doesn't really matter, this is a fairly stateless go 7046 void handleLocalConnectionComplete(IoOp* op) {} // again, irrelevant 7047 void wait_timeout() {} 7048 void fileClosed(int fd) {} // stateless so irrelevant 7049 void epoll_fd(int fd) {} 7050 } 7051 7052 /++ 7053 See [schedule] to make one of these. You then call one of the methods here to set it up: 7054 7055 --- 7056 schedule!fn(args).at(DateTime(2019, 8, 7, 12, 00, 00)); // run the function at August 7, 2019, 12 noon UTC 7057 schedule!fn(args).delay(6.seconds); // run it after waiting 6 seconds 7058 schedule!fn(args).asap(); // run it in the background as soon as the event loop gets around to it 7059 --- 7060 +/ 7061 version(with_addon_servers_connections) 7062 struct ScheduledJobHelper { 7063 private string func; 7064 private string[] args; 7065 private bool consumed; 7066 7067 private this(string func, string[] args) { 7068 this.func = func; 7069 this.args = args; 7070 } 7071 7072 ~this() { 7073 assert(consumed); 7074 } 7075 7076 /++ 7077 Schedules the job to be run at the given time. 7078 +/ 7079 void at(DateTime when, immutable TimeZone timezone = UTC()) { 7080 consumed = true; 7081 7082 auto conn = ScheduledJobServerConnection.connection; 7083 import std.file; 7084 auto st = SysTime(when, timezone); 7085 auto jobId = conn.scheduleJob(1, cast(int) st.toUnixTime(), thisExePath, func, args); 7086 } 7087 7088 /++ 7089 Schedules the job to run at least after the specified delay. 7090 +/ 7091 void delay(Duration delay) { 7092 consumed = true; 7093 7094 auto conn = ScheduledJobServerConnection.connection; 7095 import std.file; 7096 auto jobId = conn.scheduleJob(0, cast(int) delay.total!"seconds", thisExePath, func, args); 7097 } 7098 7099 /++ 7100 Runs the job in the background ASAP. 7101 7102 $(NOTE It may run in a background thread. Don't segfault!) 7103 +/ 7104 void asap() { 7105 consumed = true; 7106 7107 auto conn = ScheduledJobServerConnection.connection; 7108 import std.file; 7109 auto jobId = conn.scheduleJob(0, 1, thisExePath, func, args); 7110 } 7111 7112 /+ 7113 /++ 7114 Schedules the job to recur on the given pattern. 7115 +/ 7116 void recur(string spec) { 7117 7118 } 7119 +/ 7120 } 7121 7122 /++ 7123 First step to schedule a job on the scheduled job server. 7124 7125 The scheduled job needs to be a top-level function that doesn't read any 7126 variables from outside its arguments because it may be run in a new process, 7127 without any context existing later. 7128 7129 You MUST set details on the returned object to actually do anything! 7130 +/ 7131 template schedule(alias fn, T...) if(is(typeof(fn) == function)) { 7132 /// 7133 ScheduledJobHelper schedule(T args) { 7134 // this isn't meant to ever be called, but instead just to 7135 // get the compiler to type check the arguments passed for us 7136 auto sample = delegate() { 7137 fn(args); 7138 }; 7139 string[] sargs; 7140 foreach(arg; args) 7141 sargs ~= to!string(arg); 7142 return ScheduledJobHelper(fn.mangleof, sargs); 7143 } 7144 7145 shared static this() { 7146 scheduledJobHandlers[fn.mangleof] = delegate(string[] sargs) { 7147 import std.traits; 7148 Parameters!fn args; 7149 foreach(idx, ref arg; args) 7150 arg = to!(typeof(arg))(sargs[idx]); 7151 fn(args); 7152 }; 7153 } 7154 } 7155 7156 /// 7157 interface ScheduledJobServer { 7158 /// Use the [schedule] function for a higher-level interface. 7159 int scheduleJob(int whenIs, int when, string executable, string func, string[] args); 7160 /// 7161 void cancelJob(int jobId); 7162 } 7163 7164 version(with_addon_servers_connections) 7165 class ScheduledJobServerConnection : ScheduledJobServer { 7166 mixin ImplementRpcClientInterface!(ScheduledJobServer, "/tmp/arsd_scheduled_job_server", "--timer-server"); 7167 } 7168 7169 version(with_addon_servers) 7170 final class ScheduledJobServerImplementation : ScheduledJobServer, EventIoServer { 7171 // FIXME: we need to handle SIGCHLD in this somehow 7172 // whenIs is 0 for relative, 1 for absolute 7173 protected int scheduleJob(int whenIs, int when, string executable, string func, string[] args) { 7174 auto nj = nextJobId; 7175 nextJobId++; 7176 7177 version(linux) { 7178 import core.sys.linux.timerfd; 7179 import core.sys.linux.epoll; 7180 import core.sys.posix.unistd; 7181 7182 7183 auto fd = timerfd_create(CLOCK_REALTIME, TFD_NONBLOCK | TFD_CLOEXEC); 7184 if(fd == -1) 7185 throw new Exception("fd timer create failed"); 7186 7187 foreach(ref arg; args) 7188 arg = arg.idup; 7189 auto job = Job(executable.idup, func.idup, .dup(args), fd, nj); 7190 7191 itimerspec value; 7192 value.it_value.tv_sec = when; 7193 value.it_value.tv_nsec = 0; 7194 7195 value.it_interval.tv_sec = 0; 7196 value.it_interval.tv_nsec = 0; 7197 7198 if(timerfd_settime(fd, whenIs == 1 ? TFD_TIMER_ABSTIME : 0, &value, null) == -1) 7199 throw new Exception("couldn't set fd timer"); 7200 7201 auto op = allocateIoOp(fd, IoOp.Read, 16, (IoOp* op, int fd) { 7202 jobs.remove(nj); 7203 epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, null); 7204 close(fd); 7205 7206 7207 spawnProcess([job.executable, "--timed-job", job.func] ~ job.args); 7208 7209 return true; 7210 }); 7211 scope(failure) 7212 freeIoOp(op); 7213 7214 epoll_event ev; 7215 ev.events = EPOLLIN | EPOLLET; 7216 ev.data.ptr = op; 7217 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &ev) == -1) 7218 throw new Exception("epoll_ctl " ~ to!string(errno)); 7219 7220 jobs[nj] = job; 7221 return nj; 7222 } else assert(0); 7223 } 7224 7225 protected void cancelJob(int jobId) { 7226 version(linux) { 7227 auto job = jobId in jobs; 7228 if(job is null) 7229 return; 7230 7231 jobs.remove(jobId); 7232 7233 version(linux) { 7234 import core.sys.linux.timerfd; 7235 import core.sys.linux.epoll; 7236 import core.sys.posix.unistd; 7237 epoll_ctl(epoll_fd, EPOLL_CTL_DEL, job.timerfd, null); 7238 close(job.timerfd); 7239 } 7240 } 7241 jobs.remove(jobId); 7242 } 7243 7244 int nextJobId = 1; 7245 static struct Job { 7246 string executable; 7247 string func; 7248 string[] args; 7249 int timerfd; 7250 int id; 7251 } 7252 Job[int] jobs; 7253 7254 7255 // event io server methods below 7256 7257 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 7258 auto data = op.usedBuffer; 7259 dispatchRpcServer!ScheduledJobServer(this, data, op.fd); 7260 return false; 7261 } 7262 7263 void handleLocalConnectionClose(IoOp* op) {} // doesn't really matter, this is a fairly stateless go 7264 void handleLocalConnectionComplete(IoOp* op) {} // again, irrelevant 7265 void wait_timeout() {} 7266 void fileClosed(int fd) {} // stateless so irrelevant 7267 7268 int epoll_fd_; 7269 void epoll_fd(int fd) {this.epoll_fd_ = fd; } 7270 int epoll_fd() { return epoll_fd_; } 7271 } 7272 7273 /// 7274 version(with_addon_servers_connections) 7275 interface EventSourceServer { 7276 /++ 7277 sends this cgi request to the event server so it will be fed events. You should not do anything else with the cgi object after this. 7278 7279 $(WARNING This API is extremely unstable. I might change it or remove it without notice.) 7280 7281 See_Also: 7282 [sendEvent] 7283 +/ 7284 public static void adoptConnection(Cgi cgi, in char[] eventUrl) { 7285 /* 7286 If lastEventId is missing or empty, you just get new events as they come. 7287 7288 If it is set from something else, it sends all since then (that are still alive) 7289 down the pipe immediately. 7290 7291 The reason it can come from the header is that's what the standard defines for 7292 browser reconnects. The reason it can come from a query string is just convenience 7293 in catching up in a user-defined manner. 7294 7295 The reason the header overrides the query string is if the browser tries to reconnect, 7296 it will send the header AND the query (it reconnects to the same url), so we just 7297 want to do the restart thing. 7298 7299 Note that if you ask for "0" as the lastEventId, it will get ALL still living events. 7300 */ 7301 string lastEventId = cgi.lastEventId; 7302 if(lastEventId.length == 0 && "lastEventId" in cgi.get) 7303 lastEventId = cgi.get["lastEventId"]; 7304 7305 cgi.setResponseContentType("text/event-stream"); 7306 cgi.write(":\n", false); // to initialize the chunking and send headers before keeping the fd for later 7307 cgi.flush(); 7308 7309 cgi.closed = true; 7310 auto s = openLocalServerConnection("/tmp/arsd_cgi_event_server", "--event-server"); 7311 scope(exit) 7312 closeLocalServerConnection(s); 7313 7314 version(fastcgi) 7315 throw new Exception("sending fcgi connections not supported"); 7316 else { 7317 auto fd = cgi.getOutputFileHandle(); 7318 if(isInvalidHandle(fd)) 7319 throw new Exception("bad fd from cgi!"); 7320 7321 EventSourceServerImplementation.SendableEventConnection sec; 7322 sec.populate(cgi.responseChunked, eventUrl, lastEventId); 7323 7324 version(Posix) { 7325 auto res = write_fd(s, cast(void*) &sec, sec.sizeof, fd); 7326 assert(res == sec.sizeof); 7327 } else version(Windows) { 7328 // FIXME 7329 } 7330 } 7331 } 7332 7333 /++ 7334 Sends an event to the event server, starting it if necessary. The event server will distribute it to any listening clients, and store it for `lifetime` seconds for any later listening clients to catch up later. 7335 7336 $(WARNING This API is extremely unstable. I might change it or remove it without notice.) 7337 7338 Params: 7339 url = A string identifying this event "bucket". Listening clients must also connect to this same string. I called it `url` because I envision it being just passed as the url of the request. 7340 event = the event type string, which is used in the Javascript addEventListener API on EventSource 7341 data = the event data. Available in JS as `event.data`. 7342 lifetime = the amount of time to keep this event for replaying on the event server. 7343 7344 See_Also: 7345 [sendEventToEventServer] 7346 +/ 7347 public static void sendEvent(string url, string event, string data, int lifetime) { 7348 auto s = openLocalServerConnection("/tmp/arsd_cgi_event_server", "--event-server"); 7349 scope(exit) 7350 closeLocalServerConnection(s); 7351 7352 EventSourceServerImplementation.SendableEvent sev; 7353 sev.populate(url, event, data, lifetime); 7354 7355 version(Posix) { 7356 auto ret = send(s, &sev, sev.sizeof, 0); 7357 assert(ret == sev.sizeof); 7358 } else version(Windows) { 7359 // FIXME 7360 } 7361 } 7362 7363 /++ 7364 Messages sent to `url` will also be sent to anyone listening on `forwardUrl`. 7365 7366 See_Also: [disconnect] 7367 +/ 7368 void connect(string url, string forwardUrl); 7369 7370 /++ 7371 Disconnects `forwardUrl` from `url` 7372 7373 See_Also: [connect] 7374 +/ 7375 void disconnect(string url, string forwardUrl); 7376 } 7377 7378 /// 7379 version(with_addon_servers) 7380 final class EventSourceServerImplementation : EventSourceServer, EventIoServer { 7381 7382 protected: 7383 7384 void connect(string url, string forwardUrl) { 7385 pipes[url] ~= forwardUrl; 7386 } 7387 void disconnect(string url, string forwardUrl) { 7388 auto t = url in pipes; 7389 if(t is null) 7390 return; 7391 foreach(idx, n; (*t)) 7392 if(n == forwardUrl) { 7393 (*t)[idx] = (*t)[$-1]; 7394 (*t) = (*t)[0 .. $-1]; 7395 break; 7396 } 7397 } 7398 7399 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 7400 if(receivedFd != -1) { 7401 //writeln("GOT FD ", receivedFd, " -- ", op.usedBuffer); 7402 7403 //core.sys.posix.unistd.write(receivedFd, "hello".ptr, 5); 7404 7405 SendableEventConnection* got = cast(SendableEventConnection*) op.usedBuffer.ptr; 7406 7407 auto url = got.url.idup; 7408 eventConnectionsByUrl[url] ~= EventConnection(receivedFd, got.responseChunked > 0 ? true : false); 7409 7410 // FIXME: catch up on past messages here 7411 } else { 7412 auto data = op.usedBuffer; 7413 auto event = cast(SendableEvent*) data.ptr; 7414 7415 if(event.magic == 0xdeadbeef) { 7416 handleInputEvent(event); 7417 7418 if(event.url in pipes) 7419 foreach(pipe; pipes[event.url]) { 7420 event.url = pipe; 7421 handleInputEvent(event); 7422 } 7423 } else { 7424 dispatchRpcServer!EventSourceServer(this, data, op.fd); 7425 } 7426 } 7427 return false; 7428 } 7429 void handleLocalConnectionClose(IoOp* op) {} 7430 void handleLocalConnectionComplete(IoOp* op) {} 7431 7432 void wait_timeout() { 7433 // just keeping alive 7434 foreach(url, connections; eventConnectionsByUrl) 7435 foreach(connection; connections) 7436 if(connection.needsChunking) 7437 nonBlockingWrite(this, connection.fd, "2\r\n:\n\r\n"); 7438 else 7439 nonBlockingWrite(this, connection.fd, ":\n\r\n"); 7440 } 7441 7442 void fileClosed(int fd) { 7443 outer: foreach(url, ref connections; eventConnectionsByUrl) { 7444 foreach(idx, conn; connections) { 7445 if(fd == conn.fd) { 7446 connections[idx] = connections[$-1]; 7447 connections = connections[0 .. $ - 1]; 7448 continue outer; 7449 } 7450 } 7451 } 7452 } 7453 7454 void epoll_fd(int fd) {} 7455 7456 7457 private: 7458 7459 7460 struct SendableEventConnection { 7461 ubyte responseChunked; 7462 7463 int urlLength; 7464 char[256] urlBuffer = 0; 7465 7466 int lastEventIdLength; 7467 char[32] lastEventIdBuffer = 0; 7468 7469 char[] url() return { 7470 return urlBuffer[0 .. urlLength]; 7471 } 7472 void url(in char[] u) { 7473 urlBuffer[0 .. u.length] = u[]; 7474 urlLength = cast(int) u.length; 7475 } 7476 char[] lastEventId() return { 7477 return lastEventIdBuffer[0 .. lastEventIdLength]; 7478 } 7479 void populate(bool responseChunked, in char[] url, in char[] lastEventId) 7480 in { 7481 assert(url.length < this.urlBuffer.length); 7482 assert(lastEventId.length < this.lastEventIdBuffer.length); 7483 } 7484 do { 7485 this.responseChunked = responseChunked ? 1 : 0; 7486 this.urlLength = cast(int) url.length; 7487 this.lastEventIdLength = cast(int) lastEventId.length; 7488 7489 this.urlBuffer[0 .. url.length] = url[]; 7490 this.lastEventIdBuffer[0 .. lastEventId.length] = lastEventId[]; 7491 } 7492 } 7493 7494 struct SendableEvent { 7495 int magic = 0xdeadbeef; 7496 int urlLength; 7497 char[256] urlBuffer = 0; 7498 int typeLength; 7499 char[32] typeBuffer = 0; 7500 int messageLength; 7501 char[2048] messageBuffer = 0; 7502 int _lifetime; 7503 7504 char[] message() return { 7505 return messageBuffer[0 .. messageLength]; 7506 } 7507 char[] type() return { 7508 return typeBuffer[0 .. typeLength]; 7509 } 7510 char[] url() return { 7511 return urlBuffer[0 .. urlLength]; 7512 } 7513 void url(in char[] u) { 7514 urlBuffer[0 .. u.length] = u[]; 7515 urlLength = cast(int) u.length; 7516 } 7517 int lifetime() { 7518 return _lifetime; 7519 } 7520 7521 /// 7522 void populate(string url, string type, string message, int lifetime) 7523 in { 7524 assert(url.length < this.urlBuffer.length); 7525 assert(type.length < this.typeBuffer.length); 7526 assert(message.length < this.messageBuffer.length); 7527 } 7528 do { 7529 this.urlLength = cast(int) url.length; 7530 this.typeLength = cast(int) type.length; 7531 this.messageLength = cast(int) message.length; 7532 this._lifetime = lifetime; 7533 7534 this.urlBuffer[0 .. url.length] = url[]; 7535 this.typeBuffer[0 .. type.length] = type[]; 7536 this.messageBuffer[0 .. message.length] = message[]; 7537 } 7538 } 7539 7540 struct EventConnection { 7541 int fd; 7542 bool needsChunking; 7543 } 7544 7545 private EventConnection[][string] eventConnectionsByUrl; 7546 private string[][string] pipes; 7547 7548 private void handleInputEvent(scope SendableEvent* event) { 7549 static int eventId; 7550 7551 static struct StoredEvent { 7552 int id; 7553 string type; 7554 string message; 7555 int lifetimeRemaining; 7556 } 7557 7558 StoredEvent[][string] byUrl; 7559 7560 int thisId = ++eventId; 7561 7562 if(event.lifetime) 7563 byUrl[event.url.idup] ~= StoredEvent(thisId, event.type.idup, event.message.idup, event.lifetime); 7564 7565 auto connectionsPtr = event.url in eventConnectionsByUrl; 7566 EventConnection[] connections; 7567 if(connectionsPtr is null) 7568 return; 7569 else 7570 connections = *connectionsPtr; 7571 7572 char[4096] buffer; 7573 char[] formattedMessage; 7574 7575 void append(const char[] a) { 7576 // the 6's here are to leave room for a HTTP chunk header, if it proves necessary 7577 buffer[6 + formattedMessage.length .. 6 + formattedMessage.length + a.length] = a[]; 7578 formattedMessage = buffer[6 .. 6 + formattedMessage.length + a.length]; 7579 } 7580 7581 import std.algorithm.iteration; 7582 7583 if(connections.length) { 7584 append("id: "); 7585 append(to!string(thisId)); 7586 append("\n"); 7587 7588 append("event: "); 7589 append(event.type); 7590 append("\n"); 7591 7592 foreach(line; event.message.splitter("\n")) { 7593 append("data: "); 7594 append(line); 7595 append("\n"); 7596 } 7597 7598 append("\n"); 7599 } 7600 7601 // chunk it for HTTP! 7602 auto len = toHex(formattedMessage.length); 7603 buffer[4 .. 6] = "\r\n"[]; 7604 buffer[4 - len.length .. 4] = len[]; 7605 buffer[6 + formattedMessage.length] = '\r'; 7606 buffer[6 + formattedMessage.length + 1] = '\n'; 7607 7608 auto chunkedMessage = buffer[4 - len.length .. 6 + formattedMessage.length +2]; 7609 // done 7610 7611 // FIXME: send back requests when needed 7612 // FIXME: send a single ":\n" every 15 seconds to keep alive 7613 7614 foreach(connection; connections) { 7615 if(connection.needsChunking) { 7616 nonBlockingWrite(this, connection.fd, chunkedMessage); 7617 } else { 7618 nonBlockingWrite(this, connection.fd, formattedMessage); 7619 } 7620 } 7621 } 7622 } 7623 7624 void runAddonServer(EIS)(string localListenerName, EIS eis) if(is(EIS : EventIoServer)) { 7625 version(Posix) { 7626 7627 import core.sys.posix.unistd; 7628 import core.sys.posix.fcntl; 7629 import core.sys.posix.sys.un; 7630 7631 import core.sys.posix.signal; 7632 signal(SIGPIPE, SIG_IGN); 7633 7634 static extern(C) void sigchldhandler(int) { 7635 int status; 7636 import w = core.sys.posix.sys.wait; 7637 w.wait(&status); 7638 } 7639 signal(SIGCHLD, &sigchldhandler); 7640 7641 int sock = socket(AF_UNIX, SOCK_STREAM, 0); 7642 if(sock == -1) 7643 throw new Exception("socket " ~ to!string(errno)); 7644 7645 scope(failure) 7646 close(sock); 7647 7648 cloexec(sock); 7649 7650 // add-on server processes are assumed to be local, and thus will 7651 // use unix domain sockets. Besides, I want to pass sockets to them, 7652 // so it basically must be local (except for the session server, but meh). 7653 sockaddr_un addr; 7654 addr.sun_family = AF_UNIX; 7655 version(linux) { 7656 // on linux, we will use the abstract namespace 7657 addr.sun_path[0] = 0; 7658 addr.sun_path[1 .. localListenerName.length + 1] = cast(typeof(addr.sun_path[])) localListenerName[]; 7659 } else { 7660 // but otherwise, just use a file cuz we must. 7661 addr.sun_path[0 .. localListenerName.length] = cast(typeof(addr.sun_path[])) localListenerName[]; 7662 } 7663 7664 if(bind(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) 7665 throw new Exception("bind " ~ to!string(errno)); 7666 7667 if(listen(sock, 128) == -1) 7668 throw new Exception("listen " ~ to!string(errno)); 7669 7670 makeNonBlocking(sock); 7671 7672 version(linux) { 7673 import core.sys.linux.epoll; 7674 auto epoll_fd = epoll_create1(EPOLL_CLOEXEC); 7675 if(epoll_fd == -1) 7676 throw new Exception("epoll_create1 " ~ to!string(errno)); 7677 scope(failure) 7678 close(epoll_fd); 7679 } else { 7680 import core.sys.posix.poll; 7681 } 7682 7683 version(linux) 7684 eis.epoll_fd = epoll_fd; 7685 7686 auto acceptOp = allocateIoOp(sock, IoOp.Read, 0, null); 7687 scope(exit) 7688 freeIoOp(acceptOp); 7689 7690 version(linux) { 7691 epoll_event ev; 7692 ev.events = EPOLLIN | EPOLLET; 7693 ev.data.ptr = acceptOp; 7694 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, sock, &ev) == -1) 7695 throw new Exception("epoll_ctl " ~ to!string(errno)); 7696 7697 epoll_event[64] events; 7698 } else { 7699 pollfd[] pollfds; 7700 IoOp*[int] ioops; 7701 pollfds ~= pollfd(sock, POLLIN); 7702 ioops[sock] = acceptOp; 7703 } 7704 7705 while(true) { 7706 7707 // FIXME: it should actually do a timerfd that runs on any thing that hasn't been run recently 7708 7709 int timeout_milliseconds = 15000; // -1; // infinite 7710 //writeln("waiting for ", name); 7711 7712 version(linux) { 7713 auto nfds = epoll_wait(epoll_fd, events.ptr, events.length, timeout_milliseconds); 7714 if(nfds == -1) { 7715 if(errno == EINTR) 7716 continue; 7717 throw new Exception("epoll_wait " ~ to!string(errno)); 7718 } 7719 } else { 7720 int nfds = poll(pollfds.ptr, cast(int) pollfds.length, timeout_milliseconds); 7721 size_t lastIdx = 0; 7722 } 7723 7724 if(nfds == 0) { 7725 eis.wait_timeout(); 7726 } 7727 7728 foreach(idx; 0 .. nfds) { 7729 version(linux) { 7730 auto flags = events[idx].events; 7731 auto ioop = cast(IoOp*) events[idx].data.ptr; 7732 } else { 7733 IoOp* ioop; 7734 foreach(tidx, thing; pollfds[lastIdx .. $]) { 7735 if(thing.revents) { 7736 ioop = ioops[thing.fd]; 7737 lastIdx += tidx + 1; 7738 break; 7739 } 7740 } 7741 } 7742 7743 //writeln(flags, " ", ioop.fd); 7744 7745 void newConnection() { 7746 // on edge triggering, it is important that we get it all 7747 while(true) { 7748 auto size = cast(uint) addr.sizeof; 7749 auto ns = accept(sock, cast(sockaddr*) &addr, &size); 7750 if(ns == -1) { 7751 if(errno == EAGAIN || errno == EWOULDBLOCK) { 7752 // all done, got it all 7753 break; 7754 } 7755 throw new Exception("accept " ~ to!string(errno)); 7756 } 7757 cloexec(ns); 7758 7759 makeNonBlocking(ns); 7760 auto niop = allocateIoOp(ns, IoOp.ReadSocketHandle, 4096, &eis.handleLocalConnectionData); 7761 niop.closeHandler = &eis.handleLocalConnectionClose; 7762 niop.completeHandler = &eis.handleLocalConnectionComplete; 7763 scope(failure) freeIoOp(niop); 7764 7765 version(linux) { 7766 epoll_event nev; 7767 nev.events = EPOLLIN | EPOLLET; 7768 nev.data.ptr = niop; 7769 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, ns, &nev) == -1) 7770 throw new Exception("epoll_ctl " ~ to!string(errno)); 7771 } else { 7772 bool found = false; 7773 foreach(ref pfd; pollfds) { 7774 if(pfd.fd < 0) { 7775 pfd.fd = ns; 7776 found = true; 7777 } 7778 } 7779 if(!found) 7780 pollfds ~= pollfd(ns, POLLIN); 7781 ioops[ns] = niop; 7782 } 7783 } 7784 } 7785 7786 bool newConnectionCondition() { 7787 version(linux) 7788 return ioop.fd == sock && (flags & EPOLLIN); 7789 else 7790 return pollfds[idx].fd == sock && (pollfds[idx].revents & POLLIN); 7791 } 7792 7793 if(newConnectionCondition()) { 7794 newConnection(); 7795 } else if(ioop.operation == IoOp.ReadSocketHandle) { 7796 while(true) { 7797 int in_fd; 7798 auto got = read_fd(ioop.fd, ioop.allocatedBuffer.ptr, ioop.allocatedBuffer.length, &in_fd); 7799 if(got == -1) { 7800 if(errno == EAGAIN || errno == EWOULDBLOCK) { 7801 // all done, got it all 7802 if(ioop.completeHandler) 7803 ioop.completeHandler(ioop); 7804 break; 7805 } 7806 throw new Exception("recv " ~ to!string(errno)); 7807 } 7808 7809 if(got == 0) { 7810 if(ioop.closeHandler) { 7811 ioop.closeHandler(ioop); 7812 version(linux) {} // nothing needed 7813 else { 7814 foreach(ref pfd; pollfds) { 7815 if(pfd.fd == ioop.fd) 7816 pfd.fd = -1; 7817 } 7818 } 7819 } 7820 close(ioop.fd); 7821 freeIoOp(ioop); 7822 break; 7823 } 7824 7825 ioop.bufferLengthUsed = cast(int) got; 7826 ioop.handler(ioop, in_fd); 7827 } 7828 } else if(ioop.operation == IoOp.Read) { 7829 while(true) { 7830 auto got = read(ioop.fd, ioop.allocatedBuffer.ptr, ioop.allocatedBuffer.length); 7831 if(got == -1) { 7832 if(errno == EAGAIN || errno == EWOULDBLOCK) { 7833 // all done, got it all 7834 if(ioop.completeHandler) 7835 ioop.completeHandler(ioop); 7836 break; 7837 } 7838 throw new Exception("recv " ~ to!string(ioop.fd) ~ " errno " ~ to!string(errno)); 7839 } 7840 7841 if(got == 0) { 7842 if(ioop.closeHandler) 7843 ioop.closeHandler(ioop); 7844 close(ioop.fd); 7845 freeIoOp(ioop); 7846 break; 7847 } 7848 7849 ioop.bufferLengthUsed = cast(int) got; 7850 if(ioop.handler(ioop, ioop.fd)) { 7851 close(ioop.fd); 7852 freeIoOp(ioop); 7853 break; 7854 } 7855 } 7856 } 7857 7858 // EPOLLHUP? 7859 } 7860 } 7861 } else version(Windows) { 7862 7863 // set up a named pipe 7864 // https://msdn.microsoft.com/en-us/library/windows/desktop/ms724251(v=vs.85).aspx 7865 // https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsaduplicatesocketw 7866 // https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-getnamedpipeserverprocessid 7867 7868 } else static assert(0); 7869 } 7870 7871 7872 version(with_sendfd) 7873 // copied from the web and ported from C 7874 // see https://stackoverflow.com/questions/2358684/can-i-share-a-file-descriptor-to-another-process-on-linux-or-are-they-local-to-t 7875 ssize_t write_fd(int fd, void *ptr, size_t nbytes, int sendfd) { 7876 msghdr msg; 7877 iovec[1] iov; 7878 7879 version(OSX) { 7880 //msg.msg_accrights = cast(cattr_t) &sendfd; 7881 //msg.msg_accrightslen = int.sizeof; 7882 } else { 7883 union ControlUnion { 7884 cmsghdr cm; 7885 char[CMSG_SPACE(int.sizeof)] control; 7886 } 7887 7888 ControlUnion control_un; 7889 cmsghdr* cmptr; 7890 7891 msg.msg_control = control_un.control.ptr; 7892 msg.msg_controllen = control_un.control.length; 7893 7894 cmptr = CMSG_FIRSTHDR(&msg); 7895 cmptr.cmsg_len = CMSG_LEN(int.sizeof); 7896 cmptr.cmsg_level = SOL_SOCKET; 7897 cmptr.cmsg_type = SCM_RIGHTS; 7898 *(cast(int *) CMSG_DATA(cmptr)) = sendfd; 7899 } 7900 7901 msg.msg_name = null; 7902 msg.msg_namelen = 0; 7903 7904 iov[0].iov_base = ptr; 7905 iov[0].iov_len = nbytes; 7906 msg.msg_iov = iov.ptr; 7907 msg.msg_iovlen = 1; 7908 7909 return sendmsg(fd, &msg, 0); 7910 } 7911 7912 version(with_sendfd) 7913 // copied from the web and ported from C 7914 ssize_t read_fd(int fd, void *ptr, size_t nbytes, int *recvfd) { 7915 msghdr msg; 7916 iovec[1] iov; 7917 ssize_t n; 7918 int newfd; 7919 7920 version(OSX) { 7921 //msg.msg_accrights = cast(cattr_t) recvfd; 7922 //msg.msg_accrightslen = int.sizeof; 7923 } else { 7924 union ControlUnion { 7925 cmsghdr cm; 7926 char[CMSG_SPACE(int.sizeof)] control; 7927 } 7928 ControlUnion control_un; 7929 cmsghdr* cmptr; 7930 7931 msg.msg_control = control_un.control.ptr; 7932 msg.msg_controllen = control_un.control.length; 7933 } 7934 7935 msg.msg_name = null; 7936 msg.msg_namelen = 0; 7937 7938 iov[0].iov_base = ptr; 7939 iov[0].iov_len = nbytes; 7940 msg.msg_iov = iov.ptr; 7941 msg.msg_iovlen = 1; 7942 7943 if ( (n = recvmsg(fd, &msg, 0)) <= 0) 7944 return n; 7945 7946 version(OSX) { 7947 //if(msg.msg_accrightslen != int.sizeof) 7948 //*recvfd = -1; 7949 } else { 7950 if ( (cmptr = CMSG_FIRSTHDR(&msg)) != null && 7951 cmptr.cmsg_len == CMSG_LEN(int.sizeof)) { 7952 if (cmptr.cmsg_level != SOL_SOCKET) 7953 throw new Exception("control level != SOL_SOCKET"); 7954 if (cmptr.cmsg_type != SCM_RIGHTS) 7955 throw new Exception("control type != SCM_RIGHTS"); 7956 *recvfd = *(cast(int *) CMSG_DATA(cmptr)); 7957 } else 7958 *recvfd = -1; /* descriptor was not passed */ 7959 } 7960 7961 return n; 7962 } 7963 /* end read_fd */ 7964 7965 7966 /* 7967 Event source stuff 7968 7969 The api is: 7970 7971 sendEvent(string url, string type, string data, int timeout = 60*10); 7972 7973 attachEventListener(string url, int fd, lastId) 7974 7975 7976 It just sends to all attached listeners, and stores it until the timeout 7977 for replaying via lastEventId. 7978 */ 7979 7980 /* 7981 Session process stuff 7982 7983 it stores it all. the cgi object has a session object that can grab it 7984 7985 session may be done in the same process if possible, there is a version 7986 switch to choose if you want to override. 7987 */ 7988 7989 struct DispatcherDefinition(alias dispatchHandler, DispatcherDetails = typeof(null)) {// if(is(typeof(dispatchHandler("str", Cgi.init, void) == bool))) { // bool delegate(string urlPrefix, Cgi cgi) dispatchHandler; 7990 alias handler = dispatchHandler; 7991 string urlPrefix; 7992 bool rejectFurther; 7993 immutable(DispatcherDetails) details; 7994 } 7995 7996 private string urlify(string name) pure { 7997 return beautify(name, '-', true); 7998 } 7999 8000 private string beautify(string name, char space = ' ', bool allLowerCase = false) pure { 8001 if(name == "id") 8002 return allLowerCase ? name : "ID"; 8003 8004 char[160] buffer; 8005 int bufferIndex = 0; 8006 bool shouldCap = true; 8007 bool shouldSpace; 8008 bool lastWasCap; 8009 foreach(idx, char ch; name) { 8010 if(bufferIndex == buffer.length) return name; // out of space, just give up, not that important 8011 8012 if((ch >= 'A' && ch <= 'Z') || ch == '_') { 8013 if(lastWasCap) { 8014 // two caps in a row, don't change. Prolly acronym. 8015 } else { 8016 if(idx) 8017 shouldSpace = true; // new word, add space 8018 } 8019 8020 lastWasCap = true; 8021 } else { 8022 lastWasCap = false; 8023 } 8024 8025 if(shouldSpace) { 8026 buffer[bufferIndex++] = space; 8027 if(bufferIndex == buffer.length) return name; // out of space, just give up, not that important 8028 shouldSpace = false; 8029 } 8030 if(shouldCap) { 8031 if(ch >= 'a' && ch <= 'z') 8032 ch -= 32; 8033 shouldCap = false; 8034 } 8035 if(allLowerCase && ch >= 'A' && ch <= 'Z') 8036 ch += 32; 8037 buffer[bufferIndex++] = ch; 8038 } 8039 return buffer[0 .. bufferIndex].idup; 8040 } 8041 8042 /* 8043 string urlFor(alias func)() { 8044 return __traits(identifier, func); 8045 } 8046 */ 8047 8048 /++ 8049 UDA: The name displayed to the user in auto-generated HTML. 8050 8051 Default is `beautify(identifier)`. 8052 +/ 8053 struct DisplayName { 8054 string name; 8055 } 8056 8057 /++ 8058 UDA: The name used in the URL or web parameter. 8059 8060 Default is `urlify(identifier)` for functions and `identifier` for parameters and data members. 8061 +/ 8062 struct UrlName { 8063 string name; 8064 } 8065 8066 /++ 8067 UDA: default format to respond for this method 8068 +/ 8069 struct DefaultFormat { string value; } 8070 8071 class MissingArgumentException : Exception { 8072 string functionName; 8073 string argumentName; 8074 string argumentType; 8075 8076 this(string functionName, string argumentName, string argumentType, string file = __FILE__, size_t line = __LINE__, Throwable next = null) { 8077 this.functionName = functionName; 8078 this.argumentName = argumentName; 8079 this.argumentType = argumentType; 8080 8081 super("Missing Argument: " ~ this.argumentName, file, line, next); 8082 } 8083 } 8084 8085 /++ 8086 This can be attached to any constructor or function called from the cgi system. 8087 8088 If it is present, the function argument can NOT be set from web params, but instead 8089 is set to the return value of the given `func`. 8090 8091 If `func` can take a parameter of type [Cgi], it will be passed the one representing 8092 the current request. Otherwise, it must take zero arguments. 8093 8094 Any params in your function of type `Cgi` are automatically assumed to take the cgi object 8095 for the connection. Any of type [Session] (with an argument) is also assumed to come from 8096 the cgi object. 8097 8098 const arguments are also supported. 8099 +/ 8100 struct ifCalledFromWeb(alias func) {} 8101 8102 // it only looks at query params for GET requests, the rest must be in the body for a function argument. 8103 auto callFromCgi(alias method, T)(T dg, Cgi cgi) { 8104 8105 // FIXME: any array of structs should also be settable or gettable from csv as well. 8106 8107 // FIXME: think more about checkboxes and bools. 8108 8109 import std.traits; 8110 8111 Parameters!method params; 8112 alias idents = ParameterIdentifierTuple!method; 8113 alias defaults = ParameterDefaults!method; 8114 8115 const(string)[] names; 8116 const(string)[] values; 8117 8118 // first, check for missing arguments and initialize to defaults if necessary 8119 8120 static if(is(typeof(method) P == __parameters)) 8121 foreach(idx, param; P) {{ 8122 // see: mustNotBeSetFromWebParams 8123 static if(is(param : Cgi)) { 8124 static assert(!is(param == immutable)); 8125 cast() params[idx] = cgi; 8126 } else static if(is(param == Session!D, D)) { 8127 static assert(!is(param == immutable)); 8128 cast() params[idx] = cgi.getSessionObject!D(); 8129 } else { 8130 bool populated; 8131 foreach(uda; __traits(getAttributes, P[idx .. idx + 1])) { 8132 static if(is(uda == ifCalledFromWeb!func, alias func)) { 8133 static if(is(typeof(func(cgi)))) 8134 params[idx] = func(cgi); 8135 else 8136 params[idx] = func(); 8137 8138 populated = true; 8139 } 8140 } 8141 8142 if(!populated) { 8143 static if(__traits(compiles, { params[idx] = param.getAutomaticallyForCgi(cgi); } )) { 8144 params[idx] = param.getAutomaticallyForCgi(cgi); 8145 populated = true; 8146 } 8147 } 8148 8149 if(!populated) { 8150 auto ident = idents[idx]; 8151 if(cgi.requestMethod == Cgi.RequestMethod.GET) { 8152 if(ident !in cgi.get) { 8153 static if(is(defaults[idx] == void)) { 8154 static if(is(param == bool)) 8155 params[idx] = false; 8156 else 8157 throw new MissingArgumentException(__traits(identifier, method), ident, param.stringof); 8158 } else 8159 params[idx] = defaults[idx]; 8160 } 8161 } else { 8162 if(ident !in cgi.post) { 8163 static if(is(defaults[idx] == void)) { 8164 static if(is(param == bool)) 8165 params[idx] = false; 8166 else 8167 throw new MissingArgumentException(__traits(identifier, method), ident, param.stringof); 8168 } else 8169 params[idx] = defaults[idx]; 8170 } 8171 } 8172 } 8173 } 8174 }} 8175 8176 // second, parse the arguments in order to build up arrays, etc. 8177 8178 static bool setVariable(T)(string name, string paramName, T* what, string value) { 8179 static if(is(T == struct)) { 8180 if(name == paramName) { 8181 *what = T.init; 8182 return true; 8183 } else { 8184 // could be a child. gonna allow either obj.field OR obj[field] 8185 8186 string afterName; 8187 8188 if(name[paramName.length] == '[') { 8189 int count = 1; 8190 auto idx = paramName.length + 1; 8191 while(idx < name.length && count > 0) { 8192 if(name[idx] == '[') 8193 count++; 8194 else if(name[idx] == ']') { 8195 count--; 8196 if(count == 0) break; 8197 } 8198 idx++; 8199 } 8200 8201 if(idx == name.length) 8202 return false; // malformed 8203 8204 auto insideBrackets = name[paramName.length + 1 .. idx]; 8205 afterName = name[idx + 1 .. $]; 8206 8207 name = name[0 .. paramName.length]; 8208 8209 paramName = insideBrackets; 8210 8211 } else if(name[paramName.length] == '.') { 8212 paramName = name[paramName.length + 1 .. $]; 8213 name = paramName; 8214 int p = 0; 8215 foreach(ch; paramName) { 8216 if(ch == '.' || ch == '[') 8217 break; 8218 p++; 8219 } 8220 8221 afterName = paramName[p .. $]; 8222 paramName = paramName[0 .. p]; 8223 } else { 8224 return false; 8225 } 8226 8227 if(paramName.length) 8228 // set the child member 8229 switch(paramName) { 8230 foreach(idx, memberName; __traits(allMembers, T)) 8231 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 8232 // data member! 8233 case memberName: 8234 return setVariable(name ~ afterName, paramName, &(__traits(getMember, *what, memberName)), value); 8235 } 8236 default: 8237 // ok, not a member 8238 } 8239 } 8240 8241 return false; 8242 } else static if(isSomeString!T || isIntegral!T || isFloatingPoint!T) { 8243 *what = to!T(value); 8244 return true; 8245 } else static if(is(T == bool)) { 8246 *what = value == "1" || value == "yes" || value == "t" || value == "true" || value == "on"; 8247 return true; 8248 } else static if(is(T == K[], K)) { 8249 K tmp; 8250 if(name == paramName) { 8251 // direct - set and append 8252 if(setVariable(name, paramName, &tmp, value)) { 8253 (*what) ~= tmp; 8254 return true; 8255 } else { 8256 return false; 8257 } 8258 } else { 8259 // child, append to last element 8260 // FIXME: what about range violations??? 8261 auto ptr = &(*what)[(*what).length - 1]; 8262 return setVariable(name, paramName, ptr, value); 8263 8264 } 8265 } else static if(is(T == V[K], K, V)) { 8266 // assoc array, name[key] is valid 8267 if(name == paramName) { 8268 // no action necessary 8269 return true; 8270 } else if(name[paramName.length] == '[') { 8271 int count = 1; 8272 auto idx = paramName.length + 1; 8273 while(idx < name.length && count > 0) { 8274 if(name[idx] == '[') 8275 count++; 8276 else if(name[idx] == ']') { 8277 count--; 8278 if(count == 0) break; 8279 } 8280 idx++; 8281 } 8282 if(idx == name.length) 8283 return false; // malformed 8284 8285 auto insideBrackets = name[paramName.length + 1 .. idx]; 8286 auto afterName = name[idx + 1 .. $]; 8287 8288 auto k = to!K(insideBrackets); 8289 V v; 8290 if(auto ptr = k in *what) 8291 v = *ptr; 8292 8293 name = name[0 .. paramName.length]; 8294 //writeln(name, afterName, " ", paramName); 8295 8296 auto ret = setVariable(name ~ afterName, paramName, &v, value); 8297 if(ret) { 8298 (*what)[k] = v; 8299 return true; 8300 } 8301 } 8302 8303 return false; 8304 } else { 8305 static assert(0, "unsupported type for cgi call " ~ T.stringof); 8306 } 8307 8308 //return false; 8309 } 8310 8311 void setArgument(string name, string value) { 8312 int p; 8313 foreach(ch; name) { 8314 if(ch == '.' || ch == '[') 8315 break; 8316 p++; 8317 } 8318 8319 auto paramName = name[0 .. p]; 8320 8321 sw: switch(paramName) { 8322 static if(is(typeof(method) P == __parameters)) 8323 foreach(idx, param; P) { 8324 static if(mustNotBeSetFromWebParams!(P[idx], __traits(getAttributes, P[idx .. idx + 1]))) { 8325 // cannot be set from the outside 8326 } else { 8327 case idents[idx]: 8328 static if(is(param == Cgi.UploadedFile)) { 8329 params[idx] = cgi.files[name]; 8330 } else { 8331 setVariable(name, paramName, ¶ms[idx], value); 8332 } 8333 break sw; 8334 } 8335 } 8336 default: 8337 // ignore; not relevant argument 8338 } 8339 } 8340 8341 if(cgi.requestMethod == Cgi.RequestMethod.GET) { 8342 names = cgi.allGetNamesInOrder; 8343 values = cgi.allGetValuesInOrder; 8344 } else { 8345 names = cgi.allPostNamesInOrder; 8346 values = cgi.allPostValuesInOrder; 8347 } 8348 8349 foreach(idx, name; names) { 8350 setArgument(name, values[idx]); 8351 } 8352 8353 static if(is(ReturnType!method == void)) { 8354 typeof(null) ret; 8355 dg(params); 8356 } else { 8357 auto ret = dg(params); 8358 } 8359 8360 // FIXME: format return values 8361 // options are: json, html, csv. 8362 // also may need to wrap in envelope format: none, html, or json. 8363 return ret; 8364 } 8365 8366 private bool mustNotBeSetFromWebParams(T, attrs...)() { 8367 static if(is(T : const(Cgi))) { 8368 return true; 8369 } else static if(is(T : const(Session!D), D)) { 8370 return true; 8371 } else static if(__traits(compiles, T.getAutomaticallyForCgi(Cgi.init))) { 8372 return true; 8373 } else { 8374 foreach(uda; attrs) 8375 static if(is(uda == ifCalledFromWeb!func, alias func)) 8376 return true; 8377 return false; 8378 } 8379 } 8380 8381 private bool hasIfCalledFromWeb(attrs...)() { 8382 foreach(uda; attrs) 8383 static if(is(uda == ifCalledFromWeb!func, alias func)) 8384 return true; 8385 return false; 8386 } 8387 8388 /++ 8389 Implies POST path for the thing itself, then GET will get the automatic form. 8390 8391 The given customizer, if present, will be called as a filter on the Form object. 8392 8393 History: 8394 Added December 27, 2020 8395 +/ 8396 template AutomaticForm(alias customizer) { } 8397 8398 /+ 8399 Argument conversions: for the most part, it is to!Thing(string). 8400 8401 But arrays and structs are a bit different. Arrays come from the cgi array. Thus 8402 they are passed 8403 8404 arr=foo&arr=bar <-- notice the same name. 8405 8406 Structs are first declared with an empty thing, then have their members set individually, 8407 with dot notation. The members are not required, just the initial declaration. 8408 8409 struct Foo { 8410 int a; 8411 string b; 8412 } 8413 void test(Foo foo){} 8414 8415 foo&foo.a=5&foo.b=str <-- the first foo declares the arg, the others set the members 8416 8417 Arrays of structs use this declaration. 8418 8419 void test(Foo[] foo) {} 8420 8421 foo&foo.a=5&foo.b=bar&foo&foo.a=9 8422 8423 You can use a hidden input field in HTML forms to achieve this. The value of the naked name 8424 declaration is ignored. 8425 8426 Mind that order matters! The declaration MUST come first in the string. 8427 8428 Arrays of struct members follow this rule recursively. 8429 8430 struct Foo { 8431 int[] a; 8432 } 8433 8434 foo&foo.a=1&foo.a=2&foo&foo.a=1 8435 8436 8437 Associative arrays are formatted with brackets, after a declaration, like structs: 8438 8439 foo&foo[key]=value&foo[other_key]=value 8440 8441 8442 Note: for maximum compatibility with outside code, keep your types simple. Some libraries 8443 do not support the strict ordering requirements to work with these struct protocols. 8444 8445 FIXME: also perhaps accept application/json to better work with outside trash. 8446 8447 8448 Return values are also auto-formatted according to user-requested type: 8449 for json, it loops over and converts. 8450 for html, basic types are strings. Arrays are <ol>. Structs are <dl>. Arrays of structs are tables! 8451 +/ 8452 8453 /++ 8454 A web presenter is responsible for rendering things to HTML to be usable 8455 in a web browser. 8456 8457 They are passed as template arguments to the base classes of [WebObject] 8458 8459 Responsible for displaying stuff as HTML. You can put this into your own aggregate 8460 and override it. Use forwarding and specialization to customize it. 8461 8462 When you inherit from it, pass your own class as the CRTP argument. This lets the base 8463 class templates and your overridden templates work with each other. 8464 8465 --- 8466 class MyPresenter : WebPresenter!(MyPresenter) { 8467 @Override 8468 void presentSuccessfulReturnAsHtml(T : CustomType)(Cgi cgi, T ret, typeof(null) meta) { 8469 // present the CustomType 8470 } 8471 @Override 8472 void presentSuccessfulReturnAsHtml(T)(Cgi cgi, T ret, typeof(null) meta) { 8473 // handle everything else via the super class, which will call 8474 // back to your class when appropriate 8475 super.presentSuccessfulReturnAsHtml(cgi, ret); 8476 } 8477 } 8478 --- 8479 8480 The meta argument in there can be overridden by your own facility. 8481 8482 +/ 8483 class WebPresenter(CRTP) { 8484 8485 /// A UDA version of the built-in `override`, to be used for static template polymorphism 8486 /// If you override a plain method, use `override`. If a template, use `@Override`. 8487 enum Override; 8488 8489 string script() { 8490 return ` 8491 `; 8492 } 8493 8494 string style() { 8495 return ` 8496 :root { 8497 --mild-border: #ccc; 8498 --middle-border: #999; 8499 --accent-color: #f2f2f2; 8500 --sidebar-color: #fefefe; 8501 } 8502 ` ~ genericFormStyling() ~ genericSiteStyling(); 8503 } 8504 8505 string genericFormStyling() { 8506 return 8507 q"css 8508 table.automatic-data-display { 8509 border-collapse: collapse; 8510 border: solid 1px var(--mild-border); 8511 } 8512 8513 table.automatic-data-display td { 8514 vertical-align: top; 8515 border: solid 1px var(--mild-border); 8516 padding: 2px 4px; 8517 } 8518 8519 table.automatic-data-display th { 8520 border: solid 1px var(--mild-border); 8521 border-bottom: solid 1px var(--middle-border); 8522 padding: 2px 4px; 8523 } 8524 8525 ol.automatic-data-display { 8526 margin: 0px; 8527 list-style-position: inside; 8528 padding: 0px; 8529 } 8530 8531 dl.automatic-data-display { 8532 8533 } 8534 8535 .automatic-form { 8536 max-width: 600px; 8537 } 8538 8539 .form-field { 8540 margin: 0.5em; 8541 padding-left: 0.5em; 8542 } 8543 8544 .label-text { 8545 display: block; 8546 font-weight: bold; 8547 margin-left: -0.5em; 8548 } 8549 8550 .submit-button-holder { 8551 padding-left: 2em; 8552 } 8553 8554 .add-array-button { 8555 8556 } 8557 css"; 8558 } 8559 8560 string genericSiteStyling() { 8561 return 8562 q"css 8563 * { box-sizing: border-box; } 8564 html, body { margin: 0px; } 8565 body { 8566 font-family: sans-serif; 8567 } 8568 header { 8569 background: var(--accent-color); 8570 height: 64px; 8571 } 8572 footer { 8573 background: var(--accent-color); 8574 height: 64px; 8575 } 8576 #site-container { 8577 display: flex; 8578 } 8579 main { 8580 flex: 1 1 auto; 8581 order: 2; 8582 min-height: calc(100vh - 64px - 64px); 8583 padding: 4px; 8584 padding-left: 1em; 8585 } 8586 #sidebar { 8587 flex: 0 0 16em; 8588 order: 1; 8589 background: var(--sidebar-color); 8590 } 8591 css"; 8592 } 8593 8594 import arsd.dom; 8595 Element htmlContainer() { 8596 auto document = new Document(q"html 8597 <!DOCTYPE html> 8598 <html> 8599 <head> 8600 <title>D Application</title> 8601 <link rel="stylesheet" href="style.css" /> 8602 </head> 8603 <body> 8604 <header></header> 8605 <div id="site-container"> 8606 <main></main> 8607 <div id="sidebar"></div> 8608 </div> 8609 <footer></footer> 8610 <script src="script.js"></script> 8611 </body> 8612 </html> 8613 html", true, true); 8614 8615 return document.requireSelector("main"); 8616 } 8617 8618 /// Renders a response as an HTTP error 8619 void renderBasicError(Cgi cgi, int httpErrorCode) { 8620 cgi.setResponseStatus(getHttpCodeText(httpErrorCode)); 8621 auto c = htmlContainer(); 8622 c.innerText = getHttpCodeText(httpErrorCode); 8623 cgi.setResponseContentType("text/html; charset=utf-8"); 8624 cgi.write(c.parentDocument.toString(), true); 8625 } 8626 8627 template methodMeta(alias method) { 8628 enum methodMeta = null; 8629 } 8630 8631 void presentSuccessfulReturn(T, Meta)(Cgi cgi, T ret, Meta meta, string format) { 8632 // FIXME? format? 8633 (cast(CRTP) this).presentSuccessfulReturnAsHtml(cgi, ret, meta); 8634 } 8635 8636 /// typeof(null) (which is also used to represent functions returning `void`) do nothing 8637 /// in the default presenter - allowing the function to have full low-level control over the 8638 /// response. 8639 void presentSuccessfulReturn(T : typeof(null), Meta)(Cgi cgi, T ret, Meta meta, string format) { 8640 // nothing intentionally! 8641 } 8642 8643 /// Redirections are forwarded to [Cgi.setResponseLocation] 8644 void presentSuccessfulReturn(T : Redirection, Meta)(Cgi cgi, T ret, Meta meta, string format) { 8645 cgi.setResponseLocation(ret.to, true, getHttpCodeText(ret.code)); 8646 } 8647 8648 /// Multiple responses deconstruct the algebraic type and forward to the appropriate handler at runtime 8649 void presentSuccessfulReturn(T : MultipleResponses!Types, Meta, Types...)(Cgi cgi, T ret, Meta meta, string format) { 8650 bool outputted = false; 8651 foreach(index, type; Types) { 8652 if(ret.contains == index) { 8653 assert(!outputted); 8654 outputted = true; 8655 (cast(CRTP) this).presentSuccessfulReturn(cgi, ret.payload[index], meta, format); 8656 } 8657 } 8658 if(!outputted) 8659 assert(0); 8660 } 8661 8662 /// An instance of the [arsd.dom.FileResource] interface has its own content type; assume it is a download of some sort. 8663 void presentSuccessfulReturn(T : FileResource, Meta)(Cgi cgi, T ret, Meta meta, string format) { 8664 cgi.setCache(true); // not necessarily true but meh 8665 cgi.setResponseContentType(ret.contentType); 8666 cgi.write(ret.getData(), true); 8667 } 8668 8669 /// And the default handler for HTML will call [formatReturnValueAsHtml] and place it inside the [htmlContainer]. 8670 void presentSuccessfulReturnAsHtml(T)(Cgi cgi, T ret, typeof(null) meta) { 8671 auto container = this.htmlContainer(); 8672 container.appendChild(formatReturnValueAsHtml(ret)); 8673 cgi.write(container.parentDocument.toString(), true); 8674 } 8675 8676 /++ 8677 If you override this, you will need to cast the exception type `t` dynamically, 8678 but can then use the template arguments here to refer back to the function. 8679 8680 `func` is an alias to the method itself, and `dg` is a callable delegate to the same 8681 method on the live object. You could, in theory, change arguments and retry, but I 8682 provide that information mostly with the expectation that you will use them to make 8683 useful forms or richer error messages for the user. 8684 +/ 8685 void presentExceptionAsHtml(alias func, T)(Cgi cgi, Throwable t, T dg) { 8686 presentExceptionAsHtmlImpl(cgi, t, createAutomaticFormForFunction!(func)(dg)); 8687 } 8688 8689 void presentExceptionAsHtmlImpl(Cgi cgi, Throwable t, Form automaticForm) { 8690 if(auto mae = cast(MissingArgumentException) t) { 8691 auto container = this.htmlContainer(); 8692 if(cgi.requestMethod == Cgi.RequestMethod.POST) 8693 container.appendChild(Element.make("p", "Argument `" ~ mae.argumentName ~ "` of type `" ~ mae.argumentType ~ "` is missing")); 8694 container.appendChild(automaticForm); 8695 8696 cgi.write(container.parentDocument.toString(), true); 8697 } else { 8698 auto container = this.htmlContainer(); 8699 8700 // import std.stdio; writeln(t.toString()); 8701 8702 container.appendChild(exceptionToElement(t)); 8703 8704 container.addChild("h4", "GET"); 8705 foreach(k, v; cgi.get) { 8706 auto deets = container.addChild("details"); 8707 deets.addChild("summary", k); 8708 deets.addChild("div", v); 8709 } 8710 8711 container.addChild("h4", "POST"); 8712 foreach(k, v; cgi.post) { 8713 auto deets = container.addChild("details"); 8714 deets.addChild("summary", k); 8715 deets.addChild("div", v); 8716 } 8717 8718 8719 if(!cgi.outputtedResponseData) 8720 cgi.setResponseStatus("500 Internal Server Error"); 8721 cgi.write(container.parentDocument.toString(), true); 8722 } 8723 } 8724 8725 Element exceptionToElement(Throwable t) { 8726 auto div = Element.make("div"); 8727 div.addClass("exception-display"); 8728 8729 div.addChild("p", t.msg); 8730 div.addChild("p", "Inner code origin: " ~ typeid(t).name ~ "@" ~ t.file ~ ":" ~ to!string(t.line)); 8731 8732 auto pre = div.addChild("pre"); 8733 string s; 8734 s = t.toString(); 8735 Element currentBox; 8736 bool on = false; 8737 foreach(line; s.splitLines) { 8738 if(!on && line.startsWith("-----")) 8739 on = true; 8740 if(!on) continue; 8741 if(line.indexOf("arsd/") != -1) { 8742 if(currentBox is null) { 8743 currentBox = pre.addChild("details"); 8744 currentBox.addChild("summary", "Framework code"); 8745 } 8746 currentBox.addChild("span", line ~ "\n"); 8747 } else { 8748 pre.addChild("span", line ~ "\n"); 8749 currentBox = null; 8750 } 8751 } 8752 8753 return div; 8754 } 8755 8756 /++ 8757 Returns an element for a particular type 8758 +/ 8759 Element elementFor(T)(string displayName, string name) { 8760 import std.traits; 8761 8762 auto div = Element.make("div"); 8763 div.addClass("form-field"); 8764 8765 static if(is(T == Cgi.UploadedFile)) { 8766 Element lbl; 8767 if(displayName !is null) { 8768 lbl = div.addChild("label"); 8769 lbl.addChild("span", displayName, "label-text"); 8770 lbl.appendText(" "); 8771 } else { 8772 lbl = div; 8773 } 8774 auto i = lbl.addChild("input", name); 8775 i.attrs.name = name; 8776 i.attrs.type = "file"; 8777 } else static if(is(T == struct)) { 8778 if(displayName !is null) 8779 div.addChild("span", displayName, "label-text"); 8780 auto fieldset = div.addChild("fieldset"); 8781 fieldset.addChild("legend", beautify(T.stringof)); // FIXME 8782 fieldset.addChild("input", name); 8783 foreach(idx, memberName; __traits(allMembers, T)) 8784 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 8785 fieldset.appendChild(elementFor!(typeof(__traits(getMember, T, memberName)))(beautify(memberName), name ~ "." ~ memberName)); 8786 } 8787 } else static if(isSomeString!T || isIntegral!T || isFloatingPoint!T) { 8788 Element lbl; 8789 if(displayName !is null) { 8790 lbl = div.addChild("label"); 8791 lbl.addChild("span", displayName, "label-text"); 8792 lbl.appendText(" "); 8793 } else { 8794 lbl = div; 8795 } 8796 auto i = lbl.addChild("input", name); 8797 i.attrs.name = name; 8798 static if(isSomeString!T) 8799 i.attrs.type = "text"; 8800 else 8801 i.attrs.type = "number"; 8802 i.attrs.value = to!string(T.init); 8803 } else static if(is(T == bool)) { 8804 Element lbl; 8805 if(displayName !is null) { 8806 lbl = div.addChild("label"); 8807 lbl.addChild("span", displayName, "label-text"); 8808 lbl.appendText(" "); 8809 } else { 8810 lbl = div; 8811 } 8812 auto i = lbl.addChild("input", name); 8813 i.attrs.type = "checkbox"; 8814 i.attrs.value = "true"; 8815 i.attrs.name = name; 8816 } else static if(is(T == Cgi.UploadedFile)) { 8817 Element lbl; 8818 if(displayName !is null) { 8819 lbl = div.addChild("label"); 8820 lbl.addChild("span", displayName, "label-text"); 8821 lbl.appendText(" "); 8822 } else { 8823 lbl = div; 8824 } 8825 auto i = lbl.addChild("input", name); 8826 i.attrs.name = name; 8827 i.attrs.type = "file"; 8828 } else static if(is(T == K[], K)) { 8829 auto templ = div.addChild("template"); 8830 templ.appendChild(elementFor!(K)(null, name)); 8831 if(displayName !is null) 8832 div.addChild("span", displayName, "label-text"); 8833 auto btn = div.addChild("button"); 8834 btn.addClass("add-array-button"); 8835 btn.attrs.type = "button"; 8836 btn.innerText = "Add"; 8837 btn.attrs.onclick = q{ 8838 var a = document.importNode(this.parentNode.firstChild.content, true); 8839 this.parentNode.insertBefore(a, this); 8840 }; 8841 } else static if(is(T == V[K], K, V)) { 8842 div.innerText = "assoc array not implemented for automatic form at this time"; 8843 } else { 8844 static assert(0, "unsupported type for cgi call " ~ T.stringof); 8845 } 8846 8847 8848 return div; 8849 } 8850 8851 /// creates a form for gathering the function's arguments 8852 Form createAutomaticFormForFunction(alias method, T)(T dg) { 8853 8854 auto form = cast(Form) Element.make("form"); 8855 8856 form.method = "POST"; // FIXME 8857 8858 form.addClass("automatic-form"); 8859 8860 string formDisplayName = beautify(__traits(identifier, method)); 8861 foreach(attr; __traits(getAttributes, method)) 8862 static if(is(typeof(attr) == DisplayName)) 8863 formDisplayName = attr.name; 8864 form.addChild("h3", formDisplayName); 8865 8866 import std.traits; 8867 8868 //Parameters!method params; 8869 //alias idents = ParameterIdentifierTuple!method; 8870 //alias defaults = ParameterDefaults!method; 8871 8872 static if(is(typeof(method) P == __parameters)) 8873 foreach(idx, _; P) {{ 8874 8875 alias param = P[idx .. idx + 1]; 8876 8877 static if(!mustNotBeSetFromWebParams!(param[0], __traits(getAttributes, param))) { 8878 string displayName = beautify(__traits(identifier, param)); 8879 foreach(attr; __traits(getAttributes, param)) 8880 static if(is(typeof(attr) == DisplayName)) 8881 displayName = attr.name; 8882 auto i = form.appendChild(elementFor!(param)(displayName, __traits(identifier, param))); 8883 if(i.querySelector("input[type=file]") !is null) 8884 form.setAttribute("enctype", "multipart/form-data"); 8885 } 8886 }} 8887 8888 form.addChild("div", Html(`<input type="submit" value="Submit" />`), "submit-button-holder"); 8889 8890 return form; 8891 } 8892 8893 /// creates a form for gathering object members (for the REST object thing right now) 8894 Form createAutomaticFormForObject(T)(T obj) { 8895 auto form = cast(Form) Element.make("form"); 8896 8897 form.addClass("automatic-form"); 8898 8899 form.addChild("h3", beautify(__traits(identifier, T))); 8900 8901 import std.traits; 8902 8903 //Parameters!method params; 8904 //alias idents = ParameterIdentifierTuple!method; 8905 //alias defaults = ParameterDefaults!method; 8906 8907 foreach(idx, memberName; __traits(derivedMembers, T)) {{ 8908 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 8909 string displayName = beautify(memberName); 8910 foreach(attr; __traits(getAttributes, __traits(getMember, T, memberName))) 8911 static if(is(typeof(attr) == DisplayName)) 8912 displayName = attr.name; 8913 form.appendChild(elementFor!(typeof(__traits(getMember, T, memberName)))(displayName, memberName)); 8914 8915 form.setValue(memberName, to!string(__traits(getMember, obj, memberName))); 8916 }}} 8917 8918 form.addChild("div", Html(`<input type="submit" value="Submit" />`), "submit-button-holder"); 8919 8920 return form; 8921 } 8922 8923 /// 8924 Element formatReturnValueAsHtml(T)(T t) { 8925 import std.traits; 8926 8927 static if(is(T == typeof(null))) { 8928 return Element.make("span"); 8929 } else static if(is(T : Element)) { 8930 return t; 8931 } else static if(is(T == MultipleResponses!Types, Types...)) { 8932 foreach(index, type; Types) { 8933 if(t.contains == index) 8934 return formatReturnValueAsHtml(t.payload[index]); 8935 } 8936 assert(0); 8937 } else static if(is(T == Paginated!E, E)) { 8938 auto e = Element.make("div").addClass("paginated-result"); 8939 e.appendChild(formatReturnValueAsHtml(t.items)); 8940 if(t.nextPageUrl.length) 8941 e.appendChild(Element.make("a", "Next Page", t.nextPageUrl)); 8942 return e; 8943 } else static if(isIntegral!T || isSomeString!T || isFloatingPoint!T) { 8944 return Element.make("span", to!string(t), "automatic-data-display"); 8945 } else static if(is(T == V[K], K, V)) { 8946 auto dl = Element.make("dl"); 8947 dl.addClass("automatic-data-display associative-array"); 8948 foreach(k, v; t) { 8949 dl.addChild("dt", to!string(k)); 8950 dl.addChild("dd", formatReturnValueAsHtml(v)); 8951 } 8952 return dl; 8953 } else static if(is(T == struct)) { 8954 auto dl = Element.make("dl"); 8955 dl.addClass("automatic-data-display struct"); 8956 8957 foreach(idx, memberName; __traits(allMembers, T)) 8958 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 8959 dl.addChild("dt", beautify(memberName)); 8960 dl.addChild("dd", formatReturnValueAsHtml(__traits(getMember, t, memberName))); 8961 } 8962 8963 return dl; 8964 } else static if(is(T == bool)) { 8965 return Element.make("span", t ? "true" : "false", "automatic-data-display"); 8966 } else static if(is(T == E[], E)) { 8967 static if(is(E : RestObject!Proxy, Proxy)) { 8968 // treat RestObject similar to struct 8969 auto table = cast(Table) Element.make("table"); 8970 table.addClass("automatic-data-display"); 8971 string[] names; 8972 foreach(idx, memberName; __traits(derivedMembers, E)) 8973 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 8974 names ~= beautify(memberName); 8975 } 8976 table.appendHeaderRow(names); 8977 8978 foreach(l; t) { 8979 auto tr = table.appendRow(); 8980 foreach(idx, memberName; __traits(derivedMembers, E)) 8981 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 8982 static if(memberName == "id") { 8983 string val = to!string(__traits(getMember, l, memberName)); 8984 tr.addChild("td", Element.make("a", val, E.stringof.toLower ~ "s/" ~ val)); // FIXME 8985 } else { 8986 tr.addChild("td", formatReturnValueAsHtml(__traits(getMember, l, memberName))); 8987 } 8988 } 8989 } 8990 8991 return table; 8992 } else static if(is(E == struct)) { 8993 // an array of structs is kinda special in that I like 8994 // having those formatted as tables. 8995 auto table = cast(Table) Element.make("table"); 8996 table.addClass("automatic-data-display"); 8997 string[] names; 8998 foreach(idx, memberName; __traits(allMembers, E)) 8999 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 9000 names ~= beautify(memberName); 9001 } 9002 table.appendHeaderRow(names); 9003 9004 foreach(l; t) { 9005 auto tr = table.appendRow(); 9006 foreach(idx, memberName; __traits(allMembers, E)) 9007 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 9008 tr.addChild("td", formatReturnValueAsHtml(__traits(getMember, l, memberName))); 9009 } 9010 } 9011 9012 return table; 9013 } else { 9014 // otherwise, I will just make a list. 9015 auto ol = Element.make("ol"); 9016 ol.addClass("automatic-data-display"); 9017 foreach(e; t) 9018 ol.addChild("li", formatReturnValueAsHtml(e)); 9019 return ol; 9020 } 9021 } else static if(is(T : Object)) { 9022 static if(is(typeof(t.toHtml()))) // FIXME: maybe i will make this an interface 9023 return Element.make("div", t.toHtml()); 9024 else 9025 return Element.make("div", t.toString()); 9026 } else static assert(0, "bad return value for cgi call " ~ T.stringof); 9027 9028 assert(0); 9029 } 9030 9031 } 9032 9033 /++ 9034 The base class for the [dispatcher] function and object support. 9035 +/ 9036 class WebObject { 9037 //protected Cgi cgi; 9038 9039 protected void initialize(Cgi cgi) { 9040 //this.cgi = cgi; 9041 } 9042 } 9043 9044 /++ 9045 Can return one of the given types, decided at runtime. The syntax 9046 is to declare all the possible types in the return value, then you 9047 can `return typeof(return)(...value...)` to construct it. 9048 9049 It has an auto-generated constructor for each value it can hold. 9050 9051 --- 9052 MultipleResponses!(Redirection, string) getData(int how) { 9053 if(how & 1) 9054 return typeof(return)(Redirection("http://dpldocs.info/")); 9055 else 9056 return typeof(return)("hi there!"); 9057 } 9058 --- 9059 9060 If you have lots of returns, you could, inside the function, `alias r = typeof(return);` to shorten it a little. 9061 +/ 9062 struct MultipleResponses(T...) { 9063 private size_t contains; 9064 private union { 9065 private T payload; 9066 } 9067 9068 static foreach(index, type; T) 9069 public this(type t) { 9070 contains = index; 9071 payload[index] = t; 9072 } 9073 9074 /++ 9075 This is primarily for testing. It is your way of getting to the response. 9076 9077 Let's say you wanted to test that one holding a Redirection and a string actually 9078 holds a string, by name of "test": 9079 9080 --- 9081 auto valueToTest = your_test_function(); 9082 9083 valueToTest.visit!( 9084 (Redirection) { assert(0); }, // got a redirection instead of a string, fail the test 9085 (string s) { assert(s == "test"); } // right value, go ahead and test it. 9086 ); 9087 --- 9088 +/ 9089 void visit(Handlers...)() { 9090 template findHandler(type, HandlersToCheck...) { 9091 static if(HandlersToCheck.length == 0) 9092 alias findHandler = void; 9093 else { 9094 static if(is(typeof(HandlersToCheck[0](type.init)))) 9095 alias findHandler = handler; 9096 else 9097 alias findHandler = findHandler!(type, HandlersToCheck[1 .. $]); 9098 } 9099 } 9100 foreach(index, type; T) { 9101 alias handler = findHandler!(type, Handlers); 9102 static if(is(handler == void)) 9103 static assert(0, "Type " ~ type.stringof ~ " was not handled by visitor"); 9104 else { 9105 if(index == contains) 9106 handler(payload[index]); 9107 } 9108 } 9109 } 9110 9111 /+ 9112 auto toArsdJsvar()() { 9113 import arsd.jsvar; 9114 return var(null); 9115 } 9116 +/ 9117 } 9118 9119 struct RawResponse { 9120 int code; 9121 string[] headers; 9122 const(ubyte)[] responseBody; 9123 } 9124 9125 /++ 9126 You can return this from [WebObject] subclasses for redirections. 9127 9128 (though note the static types means that class must ALWAYS redirect if 9129 you return this directly. You might want to return [MultipleResponses] if it 9130 can be conditional) 9131 +/ 9132 struct Redirection { 9133 string to; /// The URL to redirect to. 9134 int code = 303; /// The HTTP code to retrn. 9135 } 9136 9137 /++ 9138 Serves a class' methods, as a kind of low-state RPC over the web. To be used with [dispatcher]. 9139 9140 Usage of this function will add a dependency on [arsd.dom] and [arsd.jsvar] unless you have overriden 9141 the presenter in the dispatcher. 9142 9143 FIXME: explain this better 9144 9145 You can overload functions to a limited extent: you can provide a zero-arg and non-zero-arg function, 9146 and non-zero-arg functions can filter via UDAs for various http methods. Do not attempt other overloads, 9147 the runtime result of that is undefined. 9148 9149 A method is assumed to allow any http method unless it lists some in UDAs, in which case it is limited to only those. 9150 (this might change, like maybe i will use pure as an indicator GET is ok. idk.) 9151 9152 $(WARNING 9153 --- 9154 // legal in D, undefined runtime behavior with cgi.d, it may call either method 9155 // even if you put different URL udas on it, the current code ignores them. 9156 void foo(int a) {} 9157 void foo(string a) {} 9158 --- 9159 ) 9160 9161 See_Also: [serveRestObject], [serveStaticFile] 9162 +/ 9163 auto serveApi(T)(string urlPrefix) { 9164 assert(urlPrefix[$ - 1] == '/'); 9165 return serveApiInternal!T(urlPrefix); 9166 } 9167 9168 private string nextPieceFromSlash(ref string remainingUrl) { 9169 if(remainingUrl.length == 0) 9170 return remainingUrl; 9171 int slash = 0; 9172 while(slash < remainingUrl.length && remainingUrl[slash] != '/') // && remainingUrl[slash] != '.') 9173 slash++; 9174 9175 // I am specifically passing `null` to differentiate it vs empty string 9176 // so in your ctor, `items` means new T(null) and `items/` means new T("") 9177 auto ident = remainingUrl.length == 0 ? null : remainingUrl[0 .. slash]; 9178 // so if it is the last item, the dot can be used to load an alternative view 9179 // otherwise tho the dot is considered part of the identifier 9180 // FIXME 9181 9182 // again notice "" vs null here! 9183 if(slash == remainingUrl.length) 9184 remainingUrl = null; 9185 else 9186 remainingUrl = remainingUrl[slash + 1 .. $]; 9187 9188 return ident; 9189 } 9190 9191 enum AddTrailingSlash; 9192 9193 private auto serveApiInternal(T)(string urlPrefix) { 9194 9195 import arsd.dom; 9196 import arsd.jsvar; 9197 9198 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, immutable void* details) { 9199 string remainingUrl = cgi.pathInfo[urlPrefix.length .. $]; 9200 9201 try { 9202 // see duplicated code below by searching subresource_ctor 9203 // also see mustNotBeSetFromWebParams 9204 9205 static if(is(typeof(T.__ctor) P == __parameters)) { 9206 P params; 9207 9208 foreach(pidx, param; P) { 9209 static if(is(param : Cgi)) { 9210 static assert(!is(param == immutable)); 9211 cast() params[pidx] = cgi; 9212 } else static if(is(param == Session!D, D)) { 9213 static assert(!is(param == immutable)); 9214 cast() params[pidx] = cgi.getSessionObject!D(); 9215 9216 } else { 9217 static if(hasIfCalledFromWeb!(__traits(getAttributes, P[pidx .. pidx + 1]))) { 9218 foreach(uda; __traits(getAttributes, P[pidx .. pidx + 1])) { 9219 static if(is(uda == ifCalledFromWeb!func, alias func)) { 9220 static if(is(typeof(func(cgi)))) 9221 params[pidx] = func(cgi); 9222 else 9223 params[pidx] = func(); 9224 } 9225 } 9226 } else { 9227 9228 static if(__traits(compiles, { params[pidx] = param.getAutomaticallyForCgi(cgi); } )) { 9229 params[pidx] = param.getAutomaticallyForCgi(cgi); 9230 } else static if(is(param == string)) { 9231 auto ident = nextPieceFromSlash(remainingUrl); 9232 params[pidx] = ident; 9233 } else static assert(0, "illegal type for subresource " ~ param.stringof); 9234 } 9235 } 9236 } 9237 9238 auto obj = new T(params); 9239 } else { 9240 auto obj = new T(); 9241 } 9242 9243 return internalHandlerWithObject(obj, remainingUrl, cgi, presenter); 9244 } catch(Throwable t) { 9245 switch(cgi.request("format", "html")) { 9246 case "html": 9247 static void dummy() {} 9248 presenter.presentExceptionAsHtml!(dummy)(cgi, t, &dummy); 9249 return true; 9250 case "json": 9251 var envelope = var.emptyObject; 9252 envelope.success = false; 9253 envelope.result = null; 9254 envelope.error = t.toString(); 9255 cgi.setResponseContentType("application/json"); 9256 cgi.write(envelope.toJson(), true); 9257 return true; 9258 default: 9259 throw t; 9260 // return true; 9261 } 9262 // return true; 9263 } 9264 9265 assert(0); 9266 } 9267 9268 static bool internalHandlerWithObject(T, Presenter)(T obj, string remainingUrl, Cgi cgi, Presenter presenter) { 9269 9270 obj.initialize(cgi); 9271 9272 /+ 9273 Overload rules: 9274 Any unique combination of HTTP verb and url path can be dispatched to function overloads 9275 statically. 9276 9277 Moreover, some args vs no args can be overloaded dynamically. 9278 +/ 9279 9280 auto methodNameFromUrl = nextPieceFromSlash(remainingUrl); 9281 /+ 9282 auto orig = remainingUrl; 9283 assert(0, 9284 (orig is null ? "__null" : orig) 9285 ~ " .. " ~ 9286 (methodNameFromUrl is null ? "__null" : methodNameFromUrl)); 9287 +/ 9288 9289 if(methodNameFromUrl is null) 9290 methodNameFromUrl = "__null"; 9291 9292 string hack = to!string(cgi.requestMethod) ~ " " ~ methodNameFromUrl; 9293 9294 if(remainingUrl.length) 9295 hack ~= "/"; 9296 9297 switch(hack) { 9298 foreach(methodName; __traits(derivedMembers, T)) 9299 static if(methodName != "__ctor") 9300 foreach(idx, overload; __traits(getOverloads, T, methodName)) { 9301 static if(is(typeof(overload) P == __parameters)) 9302 static if(is(typeof(overload) R == return)) 9303 static if(__traits(getProtection, overload) == "public" || __traits(getProtection, overload) == "export") 9304 { 9305 static foreach(urlNameForMethod; urlNamesForMethod!(overload, urlify(methodName))) 9306 case urlNameForMethod: 9307 9308 static if(is(R : WebObject)) { 9309 // if it returns a WebObject, it is considered a subresource. That means the url is dispatched like the ctor above. 9310 9311 // the only argument it is allowed to take, outside of cgi, session, and set up thingies, is a single string 9312 9313 // subresource_ctor 9314 // also see mustNotBeSetFromWebParams 9315 9316 P params; 9317 9318 string ident; 9319 9320 foreach(pidx, param; P) { 9321 static if(is(param : Cgi)) { 9322 static assert(!is(param == immutable)); 9323 cast() params[pidx] = cgi; 9324 } else static if(is(param == typeof(presenter))) { 9325 cast() param[pidx] = presenter; 9326 } else static if(is(param == Session!D, D)) { 9327 static assert(!is(param == immutable)); 9328 cast() params[pidx] = cgi.getSessionObject!D(); 9329 } else { 9330 static if(hasIfCalledFromWeb!(__traits(getAttributes, P[pidx .. pidx + 1]))) { 9331 foreach(uda; __traits(getAttributes, P[pidx .. pidx + 1])) { 9332 static if(is(uda == ifCalledFromWeb!func, alias func)) { 9333 static if(is(typeof(func(cgi)))) 9334 params[pidx] = func(cgi); 9335 else 9336 params[pidx] = func(); 9337 } 9338 } 9339 } else { 9340 9341 static if(__traits(compiles, { params[pidx] = param.getAutomaticallyForCgi(cgi); } )) { 9342 params[pidx] = param.getAutomaticallyForCgi(cgi); 9343 } else static if(is(param == string)) { 9344 ident = nextPieceFromSlash(remainingUrl); 9345 if(ident is null) { 9346 // trailing slash mandated on subresources 9347 cgi.setResponseLocation(cgi.pathInfo ~ "/"); 9348 return true; 9349 } else { 9350 params[pidx] = ident; 9351 } 9352 } else static assert(0, "illegal type for subresource " ~ param.stringof); 9353 } 9354 } 9355 } 9356 9357 auto nobj = (__traits(getOverloads, obj, methodName)[idx])(ident); 9358 return internalHandlerWithObject!(typeof(nobj), Presenter)(nobj, remainingUrl, cgi, presenter); 9359 } else { 9360 // 404 it if any url left - not a subresource means we don't get to play with that! 9361 if(remainingUrl.length) 9362 return false; 9363 9364 bool automaticForm; 9365 9366 foreach(attr; __traits(getAttributes, overload)) 9367 static if(is(attr == AddTrailingSlash)) { 9368 if(remainingUrl is null) { 9369 cgi.setResponseLocation(cgi.pathInfo ~ "/"); 9370 return true; 9371 } 9372 } else static if(__traits(isSame, AutomaticForm, attr)) { 9373 automaticForm = true; 9374 } 9375 9376 /+ 9377 int zeroArgOverload = -1; 9378 int overloadCount = cast(int) __traits(getOverloads, T, methodName).length; 9379 bool calledWithZeroArgs = true; 9380 foreach(k, v; cgi.get) 9381 if(k != "format") { 9382 calledWithZeroArgs = false; 9383 break; 9384 } 9385 foreach(k, v; cgi.post) 9386 if(k != "format") { 9387 calledWithZeroArgs = false; 9388 break; 9389 } 9390 9391 // first, we need to go through and see if there is an empty one, since that 9392 // changes inside. But otherwise, all the stuff I care about can be done via 9393 // simple looping (other improper overloads might be flagged for runtime semantic check) 9394 // 9395 // an argument of type Cgi is ignored for these purposes 9396 static foreach(idx, overload; __traits(getOverloads, T, methodName)) {{ 9397 static if(is(typeof(overload) P == __parameters)) 9398 static if(P.length == 0) 9399 zeroArgOverload = cast(int) idx; 9400 else static if(P.length == 1 && is(P[0] : Cgi)) 9401 zeroArgOverload = cast(int) idx; 9402 }} 9403 // FIXME: static assert if there are multiple non-zero-arg overloads usable with a single http method. 9404 bool overloadHasBeenCalled = false; 9405 static foreach(idx, overload; __traits(getOverloads, T, methodName)) {{ 9406 bool callFunction = true; 9407 // there is a zero arg overload and this is NOT it, and we have zero args - don't call this 9408 if(overloadCount > 1 && zeroArgOverload != -1 && idx != zeroArgOverload && calledWithZeroArgs) 9409 callFunction = false; 9410 // if this is the zero-arg overload, obviously it cannot be called if we got any args. 9411 if(overloadCount > 1 && idx == zeroArgOverload && !calledWithZeroArgs) 9412 callFunction = false; 9413 9414 // FIXME: so if you just add ?foo it will give the error below even when. this might not be a great idea. 9415 9416 bool hadAnyMethodRestrictions = false; 9417 bool foundAcceptableMethod = false; 9418 foreach(attr; __traits(getAttributes, overload)) { 9419 static if(is(typeof(attr) == Cgi.RequestMethod)) { 9420 hadAnyMethodRestrictions = true; 9421 if(attr == cgi.requestMethod) 9422 foundAcceptableMethod = true; 9423 } 9424 } 9425 9426 if(hadAnyMethodRestrictions && !foundAcceptableMethod) 9427 callFunction = false; 9428 9429 /+ 9430 The overloads we really want to allow are the sane ones 9431 from the web perspective. Which is likely on HTTP verbs, 9432 for the most part, but might also be potentially based on 9433 some args vs zero args, or on argument names. Can't really 9434 do argument types very reliable through the web though; those 9435 should probably be different URLs. 9436 9437 Even names I feel is better done inside the function, so I'm not 9438 going to support that here. But the HTTP verbs and zero vs some 9439 args makes sense - it lets you define custom forms pretty easily. 9440 9441 Moreover, I'm of the opinion that empty overload really only makes 9442 sense on GET for this case. On a POST, it is just a missing argument 9443 exception and that should be handled by the presenter. But meh, I'll 9444 let the user define that, D only allows one empty arg thing anyway 9445 so the method UDAs are irrelevant. 9446 +/ 9447 if(callFunction) 9448 +/ 9449 9450 if(automaticForm && cgi.requestMethod == Cgi.RequestMethod.GET) { 9451 // Should I still show the form on a json thing? idk... 9452 auto ret = presenter.createAutomaticFormForFunction!((__traits(getOverloads, obj, methodName)[idx]))(&(__traits(getOverloads, obj, methodName)[idx])); 9453 presenter.presentSuccessfulReturn(cgi, ret, presenter.methodMeta!(__traits(getOverloads, obj, methodName)[idx]), "html"); 9454 return true; 9455 } 9456 switch(cgi.request("format", defaultFormat!overload())) { 9457 case "html": 9458 // a void return (or typeof(null) lol) means you, the user, is doing it yourself. Gives full control. 9459 try { 9460 9461 auto ret = callFromCgi!(__traits(getOverloads, obj, methodName)[idx])(&(__traits(getOverloads, obj, methodName)[idx]), cgi); 9462 presenter.presentSuccessfulReturn(cgi, ret, presenter.methodMeta!(__traits(getOverloads, obj, methodName)[idx]), "html"); 9463 } catch(Throwable t) { 9464 presenter.presentExceptionAsHtml!(__traits(getOverloads, obj, methodName)[idx])(cgi, t, &(__traits(getOverloads, obj, methodName)[idx])); 9465 } 9466 return true; 9467 case "json": 9468 auto ret = callFromCgi!(__traits(getOverloads, obj, methodName)[idx])(&(__traits(getOverloads, obj, methodName)[idx]), cgi); 9469 static if(is(typeof(ret) == MultipleResponses!Types, Types...)) { 9470 var json; 9471 foreach(index, type; Types) { 9472 if(ret.contains == index) 9473 json = ret.payload[index]; 9474 } 9475 } else { 9476 var json = ret; 9477 } 9478 var envelope = json; // var.emptyObject; 9479 /* 9480 envelope.success = true; 9481 envelope.result = json; 9482 envelope.error = null; 9483 */ 9484 cgi.setResponseContentType("application/json"); 9485 cgi.write(envelope.toJson(), true); 9486 return true; 9487 default: 9488 cgi.setResponseStatus("406 Not Acceptable"); // not exactly but sort of. 9489 return true; 9490 } 9491 //}} 9492 9493 //cgi.header("Accept: POST"); // FIXME list the real thing 9494 //cgi.setResponseStatus("405 Method Not Allowed"); // again, not exactly, but sort of. no overload matched our args, almost certainly due to http verb filtering. 9495 //return true; 9496 } 9497 } 9498 } 9499 case "GET script.js": 9500 cgi.setResponseContentType("text/javascript"); 9501 cgi.gzipResponse = true; 9502 cgi.write(presenter.script(), true); 9503 return true; 9504 case "GET style.css": 9505 cgi.setResponseContentType("text/css"); 9506 cgi.gzipResponse = true; 9507 cgi.write(presenter.style(), true); 9508 return true; 9509 default: 9510 return false; 9511 } 9512 9513 assert(0); 9514 } 9515 return DispatcherDefinition!internalHandler(urlPrefix, false); 9516 } 9517 9518 string defaultFormat(alias method)() { 9519 bool nonConstConditionForWorkingAroundASpuriousDmdWarning = true; 9520 foreach(attr; __traits(getAttributes, method)) { 9521 static if(is(typeof(attr) == DefaultFormat)) { 9522 if(nonConstConditionForWorkingAroundASpuriousDmdWarning) 9523 return attr.value; 9524 } 9525 } 9526 return "html"; 9527 } 9528 9529 struct Paginated(T) { 9530 T[] items; 9531 string nextPageUrl; 9532 } 9533 9534 template urlNamesForMethod(alias method, string default_) { 9535 string[] helper() { 9536 auto verb = Cgi.RequestMethod.GET; 9537 bool foundVerb = false; 9538 bool foundNoun = false; 9539 9540 string def = default_; 9541 9542 bool hasAutomaticForm = false; 9543 9544 foreach(attr; __traits(getAttributes, method)) { 9545 static if(is(typeof(attr) == Cgi.RequestMethod)) { 9546 verb = attr; 9547 if(foundVerb) 9548 assert(0, "Multiple http verbs on one function is not currently supported"); 9549 foundVerb = true; 9550 } 9551 static if(is(typeof(attr) == UrlName)) { 9552 if(foundNoun) 9553 assert(0, "Multiple url names on one function is not currently supported"); 9554 foundNoun = true; 9555 def = attr.name; 9556 } 9557 static if(__traits(isSame, attr, AutomaticForm)) { 9558 hasAutomaticForm = true; 9559 } 9560 } 9561 9562 if(def is null) 9563 def = "__null"; 9564 9565 string[] ret; 9566 9567 static if(is(typeof(method) R == return)) { 9568 static if(is(R : WebObject)) { 9569 def ~= "/"; 9570 foreach(v; __traits(allMembers, Cgi.RequestMethod)) 9571 ret ~= v ~ " " ~ def; 9572 } else { 9573 if(hasAutomaticForm) { 9574 ret ~= "GET " ~ def; 9575 ret ~= "POST " ~ def; 9576 } else { 9577 ret ~= to!string(verb) ~ " " ~ def; 9578 } 9579 } 9580 } else static assert(0); 9581 9582 return ret; 9583 } 9584 enum urlNamesForMethod = helper(); 9585 } 9586 9587 9588 enum AccessCheck { 9589 allowed, 9590 denied, 9591 nonExistant, 9592 } 9593 9594 enum Operation { 9595 show, 9596 create, 9597 replace, 9598 remove, 9599 update 9600 } 9601 9602 enum UpdateResult { 9603 accessDenied, 9604 noSuchResource, 9605 success, 9606 failure, 9607 unnecessary 9608 } 9609 9610 enum ValidationResult { 9611 valid, 9612 invalid 9613 } 9614 9615 9616 /++ 9617 The base of all REST objects, to be used with [serveRestObject] and [serveRestCollectionOf]. 9618 9619 WARNING: this is not stable. 9620 +/ 9621 class RestObject(CRTP) : WebObject { 9622 9623 import arsd.dom; 9624 import arsd.jsvar; 9625 9626 /// Prepare the object to be shown. 9627 void show() {} 9628 /// ditto 9629 void show(string urlId) { 9630 load(urlId); 9631 show(); 9632 } 9633 9634 /// Override this to provide access control to this object. 9635 AccessCheck accessCheck(string urlId, Operation operation) { 9636 return AccessCheck.allowed; 9637 } 9638 9639 ValidationResult validate() { 9640 // FIXME 9641 return ValidationResult.valid; 9642 } 9643 9644 string getUrlSlug() { 9645 import std.conv; 9646 static if(is(typeof(CRTP.id))) 9647 return to!string((cast(CRTP) this).id); 9648 else 9649 return null; 9650 } 9651 9652 // The functions with more arguments are the low-level ones, 9653 // they forward to the ones with fewer arguments by default. 9654 9655 // POST on a parent collection - this is called from a collection class after the members are updated 9656 /++ 9657 Given a populated object, this creates a new entry. Returns the url identifier 9658 of the new object. 9659 +/ 9660 string create(scope void delegate() applyChanges) { 9661 applyChanges(); 9662 save(); 9663 return getUrlSlug(); 9664 } 9665 9666 void replace() { 9667 save(); 9668 } 9669 void replace(string urlId, scope void delegate() applyChanges) { 9670 load(urlId); 9671 applyChanges(); 9672 replace(); 9673 } 9674 9675 void update(string[] fieldList) { 9676 save(); 9677 } 9678 void update(string urlId, scope void delegate() applyChanges, string[] fieldList) { 9679 load(urlId); 9680 applyChanges(); 9681 update(fieldList); 9682 } 9683 9684 void remove() {} 9685 9686 void remove(string urlId) { 9687 load(urlId); 9688 remove(); 9689 } 9690 9691 abstract void load(string urlId); 9692 abstract void save(); 9693 9694 Element toHtml(Presenter)(Presenter presenter) { 9695 import arsd.dom; 9696 import std.conv; 9697 auto obj = cast(CRTP) this; 9698 auto div = Element.make("div"); 9699 div.addClass("Dclass_" ~ CRTP.stringof); 9700 div.dataset.url = getUrlSlug(); 9701 bool first = true; 9702 foreach(idx, memberName; __traits(derivedMembers, CRTP)) 9703 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 9704 if(!first) div.addChild("br"); else first = false; 9705 div.appendChild(presenter.formatReturnValueAsHtml(__traits(getMember, obj, memberName))); 9706 } 9707 return div; 9708 } 9709 9710 var toJson() { 9711 import arsd.jsvar; 9712 var v = var.emptyObject(); 9713 auto obj = cast(CRTP) this; 9714 foreach(idx, memberName; __traits(derivedMembers, CRTP)) 9715 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 9716 v[memberName] = __traits(getMember, obj, memberName); 9717 } 9718 return v; 9719 } 9720 9721 /+ 9722 auto structOf(this This) { 9723 9724 } 9725 +/ 9726 } 9727 9728 // FIXME XSRF token, prolly can just put in a cookie and then it needs to be copied to header or form hidden value 9729 // https://use-the-index-luke.com/sql/partial-results/fetch-next-page 9730 9731 /++ 9732 Base class for REST collections. 9733 +/ 9734 class CollectionOf(Obj) : RestObject!(CollectionOf) { 9735 /// You might subclass this and use the cgi object's query params 9736 /// to implement a search filter, for example. 9737 /// 9738 /// FIXME: design a way to auto-generate that form 9739 /// (other than using the WebObject thing above lol 9740 // it'll prolly just be some searchParams UDA or maybe an enum. 9741 // 9742 // pagination too perhaps. 9743 // 9744 // and sorting too 9745 IndexResult index() { return IndexResult.init; } 9746 9747 string[] sortableFields() { return null; } 9748 string[] searchableFields() { return null; } 9749 9750 struct IndexResult { 9751 Obj[] results; 9752 9753 string[] sortableFields; 9754 9755 string previousPageIdentifier; 9756 string nextPageIdentifier; 9757 string firstPageIdentifier; 9758 string lastPageIdentifier; 9759 9760 int numberOfPages; 9761 } 9762 9763 override string create(scope void delegate() applyChanges) { assert(0); } 9764 override void load(string urlId) { assert(0); } 9765 override void save() { assert(0); } 9766 override void show() { 9767 index(); 9768 } 9769 override void show(string urlId) { 9770 show(); 9771 } 9772 9773 /// Proxy POST requests (create calls) to the child collection 9774 alias PostProxy = Obj; 9775 } 9776 9777 /++ 9778 Serves a REST object, similar to a Ruby on Rails resource. 9779 9780 You put data members in your class. cgi.d will automatically make something out of those. 9781 9782 It will call your constructor with the ID from the URL. This may be null. 9783 It will then populate the data members from the request. 9784 It will then call a method, if present, telling what happened. You don't need to write these! 9785 It finally returns a reply. 9786 9787 Your methods are passed a list of fields it actually set. 9788 9789 The URL mapping - despite my general skepticism of the wisdom - matches up with what most REST 9790 APIs I have used seem to follow. (I REALLY want to put trailing slashes on it though. Works better 9791 with relative linking. But meh.) 9792 9793 GET /items -> index. all values not set. 9794 GET /items/id -> get. only ID will be set, other params ignored. 9795 POST /items -> create. values set as given 9796 PUT /items/id -> replace. values set as given 9797 or POST /items/id with cgi.post["_method"] (thus urlencoded or multipart content-type) set to "PUT" to work around browser/html limitation 9798 a GET with cgi.get["_method"] (in the url) set to "PUT" will render a form. 9799 PATCH /items/id -> update. values set as given, list of changed fields passed 9800 or POST /items/id with cgi.post["_method"] == "PATCH" 9801 DELETE /items/id -> destroy. only ID guaranteed to be set 9802 or POST /items/id with cgi.post["_method"] == "DELETE" 9803 9804 Following the stupid convention, there will never be a trailing slash here, and if it is there, it will 9805 redirect you away from it. 9806 9807 API clients should set the `Accept` HTTP header to application/json or the cgi.get["_format"] = "json" var. 9808 9809 I will also let you change the default, if you must. 9810 9811 // One add-on is validation. You can issue a HTTP GET to a resource with _method = VALIDATE to check potential changes. 9812 9813 You can define sub-resources on your object inside the object. These sub-resources are also REST objects 9814 that follow the same thing. They may be individual resources or collections themselves. 9815 9816 Your class is expected to have at least the following methods: 9817 9818 FIXME: i kinda wanna add a routes object to the initialize call 9819 9820 create 9821 Create returns the new address on success, some code on failure. 9822 show 9823 index 9824 update 9825 remove 9826 9827 You will want to be able to customize the HTTP, HTML, and JSON returns but generally shouldn't have to - the defaults 9828 should usually work. The returned JSON will include a field "href" on all returned objects along with "id". Or omething like that. 9829 9830 Usage of this function will add a dependency on [arsd.dom] and [arsd.jsvar]. 9831 9832 NOT IMPLEMENTED 9833 9834 9835 Really, a collection is a resource with a bunch of subresources. 9836 9837 GET /items 9838 index because it is GET on the top resource 9839 9840 GET /items/foo 9841 item but different than items? 9842 9843 class Items { 9844 9845 } 9846 9847 ... but meh, a collection can be automated. not worth making it 9848 a separate thing, let's look at a real example. Users has many 9849 items and a virtual one, /users/current. 9850 9851 the individual users have properties and two sub-resources: 9852 session, which is just one, and comments, a collection. 9853 9854 class User : RestObject!() { // no parent 9855 int id; 9856 string name; 9857 9858 // the default implementations of the urlId ones is to call load(that_id) then call the arg-less one. 9859 // but you can override them to do it differently. 9860 9861 // any member which is of type RestObject can be linked automatically via href btw. 9862 9863 void show() {} 9864 void show(string urlId) {} // automated! GET of this specific thing 9865 void create() {} // POST on a parent collection - this is called from a collection class after the members are updated 9866 void replace(string urlId) {} // this is the PUT; really, it just updates all fields. 9867 void update(string urlId, string[] fieldList) {} // PATCH, it updates some fields. 9868 void remove(string urlId) {} // DELETE 9869 9870 void load(string urlId) {} // the default implementation of show() populates the id, then 9871 9872 this() {} 9873 9874 mixin Subresource!Session; 9875 mixin Subresource!Comment; 9876 } 9877 9878 class Session : RestObject!() { 9879 // the parent object may not be fully constructed/loaded 9880 this(User parent) {} 9881 9882 } 9883 9884 class Comment : CollectionOf!Comment { 9885 this(User parent) {} 9886 } 9887 9888 class Users : CollectionOf!User { 9889 // but you don't strictly need ANYTHING on a collection; it will just... collect. Implement the subobjects. 9890 void index() {} // GET on this specific thing; just like show really, just different name for the different semantics. 9891 User create() {} // You MAY implement this, but the default is to create a new object, populate it from args, and then call create() on the child 9892 } 9893 9894 +/ 9895 auto serveRestObject(T)(string urlPrefix) { 9896 assert(urlPrefix[0] == '/'); 9897 assert(urlPrefix[$ - 1] != '/', "Do NOT use a trailing slash on REST objects."); 9898 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, immutable void* details) { 9899 string url = cgi.pathInfo[urlPrefix.length .. $]; 9900 9901 if(url.length && url[$ - 1] == '/') { 9902 // remove the final slash... 9903 cgi.setResponseLocation(cgi.scriptName ~ cgi.pathInfo[0 .. $ - 1]); 9904 return true; 9905 } 9906 9907 return restObjectServeHandler!T(cgi, presenter, url); 9908 } 9909 return DispatcherDefinition!internalHandler(urlPrefix, false); 9910 } 9911 9912 /+ 9913 /// Convenience method for serving a collection. It will be named the same 9914 /// as type T, just with an s at the end. If you need any further, just 9915 /// write the class yourself. 9916 auto serveRestCollectionOf(T)(string urlPrefix) { 9917 assert(urlPrefix[0] == '/'); 9918 mixin(`static class `~T.stringof~`s : CollectionOf!(T) {}`); 9919 return serveRestObject!(mixin(T.stringof ~ "s"))(urlPrefix); 9920 } 9921 +/ 9922 9923 bool restObjectServeHandler(T, Presenter)(Cgi cgi, Presenter presenter, string url) { 9924 string urlId = null; 9925 if(url.length && url[0] == '/') { 9926 // asking for a subobject 9927 urlId = url[1 .. $]; 9928 foreach(idx, ch; urlId) { 9929 if(ch == '/') { 9930 urlId = urlId[0 .. idx]; 9931 break; 9932 } 9933 } 9934 } 9935 9936 // FIXME handle other subresources 9937 9938 static if(is(T : CollectionOf!(C), C)) { 9939 if(urlId !is null) { 9940 return restObjectServeHandler!(C, Presenter)(cgi, presenter, url); // FIXME? urlId); 9941 } 9942 } 9943 9944 // FIXME: support precondition failed, if-modified-since, expectation failed, etc. 9945 9946 auto obj = new T(); 9947 obj.initialize(cgi); 9948 // FIXME: populate reflection info delegates 9949 9950 9951 // FIXME: I am not happy with this. 9952 switch(urlId) { 9953 case "script.js": 9954 cgi.setResponseContentType("text/javascript"); 9955 cgi.gzipResponse = true; 9956 cgi.write(presenter.script(), true); 9957 return true; 9958 case "style.css": 9959 cgi.setResponseContentType("text/css"); 9960 cgi.gzipResponse = true; 9961 cgi.write(presenter.style(), true); 9962 return true; 9963 default: 9964 // intentionally blank 9965 } 9966 9967 9968 9969 9970 static void applyChangesTemplate(Obj)(Cgi cgi, Obj obj) { 9971 foreach(idx, memberName; __traits(derivedMembers, Obj)) 9972 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 9973 __traits(getMember, obj, memberName) = cgi.request(memberName, __traits(getMember, obj, memberName)); 9974 } 9975 } 9976 void applyChanges() { 9977 applyChangesTemplate(cgi, obj); 9978 } 9979 9980 string[] modifiedList; 9981 9982 void writeObject(bool addFormLinks) { 9983 if(cgi.request("format") == "json") { 9984 cgi.setResponseContentType("application/json"); 9985 cgi.write(obj.toJson().toString, true); 9986 } else { 9987 auto container = presenter.htmlContainer(); 9988 if(addFormLinks) { 9989 static if(is(T : CollectionOf!(C), C)) 9990 container.appendHtml(` 9991 <form> 9992 <button type="submit" name="_method" value="POST">Create New</button> 9993 </form> 9994 `); 9995 else 9996 container.appendHtml(` 9997 <a href="..">Back</a> 9998 <form> 9999 <button type="submit" name="_method" value="PATCH">Edit</button> 10000 <button type="submit" name="_method" value="DELETE">Delete</button> 10001 </form> 10002 `); 10003 } 10004 container.appendChild(obj.toHtml(presenter)); 10005 cgi.write(container.parentDocument.toString, true); 10006 } 10007 } 10008 10009 // FIXME: I think I need a set type in here.... 10010 // it will be nice to pass sets of members. 10011 10012 try 10013 switch(cgi.requestMethod) { 10014 case Cgi.RequestMethod.GET: 10015 // I could prolly use template this parameters in the implementation above for some reflection stuff. 10016 // sure, it doesn't automatically work in subclasses... but I instantiate here anyway... 10017 10018 // automatic forms here for usable basic auto site from browser. 10019 // even if the format is json, it could actually send out the links and formats, but really there i'ma be meh. 10020 switch(cgi.request("_method", "GET")) { 10021 case "GET": 10022 static if(is(T : CollectionOf!(C), C)) { 10023 auto results = obj.index(); 10024 if(cgi.request("format", "html") == "html") { 10025 auto container = presenter.htmlContainer(); 10026 auto html = presenter.formatReturnValueAsHtml(results.results); 10027 container.appendHtml(` 10028 <form> 10029 <button type="submit" name="_method" value="POST">Create New</button> 10030 </form> 10031 `); 10032 10033 container.appendChild(html); 10034 cgi.write(container.parentDocument.toString, true); 10035 } else { 10036 cgi.setResponseContentType("application/json"); 10037 import arsd.jsvar; 10038 var json = var.emptyArray; 10039 foreach(r; results.results) { 10040 var o = var.emptyObject; 10041 foreach(idx, memberName; __traits(derivedMembers, typeof(r))) 10042 static if(__traits(compiles, __traits(getMember, r, memberName).offsetof)) { 10043 o[memberName] = __traits(getMember, r, memberName); 10044 } 10045 10046 json ~= o; 10047 } 10048 cgi.write(json.toJson(), true); 10049 } 10050 } else { 10051 obj.show(urlId); 10052 writeObject(true); 10053 } 10054 break; 10055 case "PATCH": 10056 obj.load(urlId); 10057 goto case; 10058 case "PUT": 10059 case "POST": 10060 // an editing form for the object 10061 auto container = presenter.htmlContainer(); 10062 static if(__traits(compiles, () { auto o = new obj.PostProxy(); })) { 10063 auto form = (cgi.request("_method") == "POST") ? presenter.createAutomaticFormForObject(new obj.PostProxy()) : presenter.createAutomaticFormForObject(obj); 10064 } else { 10065 auto form = presenter.createAutomaticFormForObject(obj); 10066 } 10067 form.attrs.method = "POST"; 10068 form.setValue("_method", cgi.request("_method", "GET")); 10069 container.appendChild(form); 10070 cgi.write(container.parentDocument.toString(), true); 10071 break; 10072 case "DELETE": 10073 // FIXME: a delete form for the object (can be phrased "are you sure?") 10074 auto container = presenter.htmlContainer(); 10075 container.appendHtml(` 10076 <form method="POST"> 10077 Are you sure you want to delete this item? 10078 <input type="hidden" name="_method" value="DELETE" /> 10079 <input type="submit" value="Yes, Delete It" /> 10080 </form> 10081 10082 `); 10083 cgi.write(container.parentDocument.toString(), true); 10084 break; 10085 default: 10086 cgi.write("bad method\n", true); 10087 } 10088 break; 10089 case Cgi.RequestMethod.POST: 10090 // this is to allow compatibility with HTML forms 10091 switch(cgi.request("_method", "POST")) { 10092 case "PUT": 10093 goto PUT; 10094 case "PATCH": 10095 goto PATCH; 10096 case "DELETE": 10097 goto DELETE; 10098 case "POST": 10099 static if(__traits(compiles, () { auto o = new obj.PostProxy(); })) { 10100 auto p = new obj.PostProxy(); 10101 void specialApplyChanges() { 10102 applyChangesTemplate(cgi, p); 10103 } 10104 string n = p.create(&specialApplyChanges); 10105 } else { 10106 string n = obj.create(&applyChanges); 10107 } 10108 10109 auto newUrl = cgi.scriptName ~ cgi.pathInfo ~ "/" ~ n; 10110 cgi.setResponseLocation(newUrl); 10111 cgi.setResponseStatus("201 Created"); 10112 cgi.write(`The object has been created.`); 10113 break; 10114 default: 10115 cgi.write("bad method\n", true); 10116 } 10117 // FIXME this should be valid on the collection, but not the child.... 10118 // 303 See Other 10119 break; 10120 case Cgi.RequestMethod.PUT: 10121 PUT: 10122 obj.replace(urlId, &applyChanges); 10123 writeObject(false); 10124 break; 10125 case Cgi.RequestMethod.PATCH: 10126 PATCH: 10127 obj.update(urlId, &applyChanges, modifiedList); 10128 writeObject(false); 10129 break; 10130 case Cgi.RequestMethod.DELETE: 10131 DELETE: 10132 obj.remove(urlId); 10133 cgi.setResponseStatus("204 No Content"); 10134 break; 10135 default: 10136 // FIXME: OPTIONS, HEAD 10137 } 10138 catch(Throwable t) { 10139 presenter.presentExceptionAsHtml!(DUMMY)(cgi, t, null); 10140 } 10141 10142 return true; 10143 } 10144 10145 struct DUMMY {} 10146 10147 /+ 10148 struct SetOfFields(T) { 10149 private void[0][string] storage; 10150 void set(string what) { 10151 //storage[what] = 10152 } 10153 void unset(string what) {} 10154 void setAll() {} 10155 void unsetAll() {} 10156 bool isPresent(string what) { return false; } 10157 } 10158 +/ 10159 10160 /+ 10161 enum readonly; 10162 enum hideonindex; 10163 +/ 10164 10165 /++ 10166 Serves a static file. To be used with [dispatcher]. 10167 10168 See_Also: [serveApi], [serveRestObject], [dispatcher], [serveRedirect] 10169 +/ 10170 auto serveStaticFile(string urlPrefix, string filename = null, string contentType = null) { 10171 // https://baus.net/on-tcp_cork/ 10172 // man 2 sendfile 10173 assert(urlPrefix[0] == '/'); 10174 if(filename is null) 10175 filename = urlPrefix[1 .. $]; 10176 if(contentType is null) { 10177 contentType = contentTypeFromFileExtension(filename); 10178 } 10179 10180 static struct DispatcherDetails { 10181 string filename; 10182 string contentType; 10183 } 10184 10185 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 10186 if(details.contentType.indexOf("image/") == 0) 10187 cgi.setCache(true); 10188 cgi.setResponseContentType(details.contentType); 10189 cgi.write(std.file.read(details.filename), true); 10190 return true; 10191 } 10192 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, true, DispatcherDetails(filename, contentType)); 10193 } 10194 10195 /++ 10196 Serves static data. To be used with [dispatcher]. 10197 10198 History: 10199 Added October 31, 2021 10200 +/ 10201 auto serveStaticData(string urlPrefix, immutable(void)[] data, string contentType = null) { 10202 assert(urlPrefix[0] == '/'); 10203 if(contentType is null) { 10204 contentType = contentTypeFromFileExtension(urlPrefix); 10205 } 10206 10207 static struct DispatcherDetails { 10208 immutable(void)[] data; 10209 string contentType; 10210 } 10211 10212 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 10213 cgi.setCache(true); 10214 cgi.setResponseContentType(details.contentType); 10215 cgi.write(details.data, true); 10216 return true; 10217 } 10218 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, true, DispatcherDetails(data, contentType)); 10219 } 10220 10221 string contentTypeFromFileExtension(string filename) { 10222 if(filename.endsWith(".png")) 10223 return "image/png"; 10224 if(filename.endsWith(".svg")) 10225 return "image/svg+xml"; 10226 if(filename.endsWith(".jpg")) 10227 return "image/jpeg"; 10228 if(filename.endsWith(".html")) 10229 return "text/html"; 10230 if(filename.endsWith(".css")) 10231 return "text/css"; 10232 if(filename.endsWith(".js")) 10233 return "application/javascript"; 10234 if(filename.endsWith(".wasm")) 10235 return "application/wasm"; 10236 if(filename.endsWith(".mp3")) 10237 return "audio/mpeg"; 10238 return null; 10239 } 10240 10241 /// This serves a directory full of static files, figuring out the content-types from file extensions. 10242 /// It does not let you to descend into subdirectories (or ascend out of it, of course) 10243 auto serveStaticFileDirectory(string urlPrefix, string directory = null) { 10244 assert(urlPrefix[0] == '/'); 10245 assert(urlPrefix[$-1] == '/'); 10246 10247 static struct DispatcherDetails { 10248 string directory; 10249 } 10250 10251 if(directory is null) 10252 directory = urlPrefix[1 .. $]; 10253 10254 assert(directory[$-1] == '/'); 10255 10256 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 10257 auto file = cgi.pathInfo[urlPrefix.length .. $]; 10258 if(file.indexOf("/") != -1 || file.indexOf("\\") != -1) 10259 return false; 10260 10261 auto contentType = contentTypeFromFileExtension(file); 10262 10263 auto fn = details.directory ~ file; 10264 if(std.file.exists(fn)) { 10265 //if(contentType.indexOf("image/") == 0) 10266 //cgi.setCache(true); 10267 //else if(contentType.indexOf("audio/") == 0) 10268 cgi.setCache(true); 10269 cgi.setResponseContentType(contentType); 10270 cgi.write(std.file.read(fn), true); 10271 return true; 10272 } else { 10273 return false; 10274 } 10275 } 10276 10277 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, false, DispatcherDetails(directory)); 10278 } 10279 10280 /++ 10281 Redirects one url to another 10282 10283 See_Also: [dispatcher], [serveStaticFile] 10284 +/ 10285 auto serveRedirect(string urlPrefix, string redirectTo, int code = 303) { 10286 assert(urlPrefix[0] == '/'); 10287 static struct DispatcherDetails { 10288 string redirectTo; 10289 string code; 10290 } 10291 10292 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 10293 cgi.setResponseLocation(details.redirectTo, true, details.code); 10294 return true; 10295 } 10296 10297 10298 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, true, DispatcherDetails(redirectTo, getHttpCodeText(code))); 10299 } 10300 10301 /// Used exclusively with `dispatchTo` 10302 struct DispatcherData(Presenter) { 10303 Cgi cgi; /// You can use this cgi object. 10304 Presenter presenter; /// This is the presenter from top level, and will be forwarded to the sub-dispatcher. 10305 size_t pathInfoStart; /// This is forwarded to the sub-dispatcher. It may be marked private later, or at least read-only. 10306 } 10307 10308 /++ 10309 Dispatches the URL to a specific function. 10310 +/ 10311 auto handleWith(alias handler)(string urlPrefix) { 10312 // cuz I'm too lazy to do it better right now 10313 static class Hack : WebObject { 10314 static import std.traits; 10315 @UrlName("") 10316 auto handle(std.traits.Parameters!handler args) { 10317 return handler(args); 10318 } 10319 } 10320 10321 return urlPrefix.serveApiInternal!Hack; 10322 } 10323 10324 /++ 10325 Dispatches the URL (and anything under it) to another dispatcher function. The function should look something like this: 10326 10327 --- 10328 bool other(DD)(DD dd) { 10329 return dd.dispatcher!( 10330 "/whatever".serveRedirect("/success"), 10331 "/api/".serveApi!MyClass 10332 ); 10333 } 10334 --- 10335 10336 The `DD` in there will be an instance of [DispatcherData] which you can inspect, or forward to another dispatcher 10337 here. It is a template to account for any Presenter type, so you can do compile-time analysis in your presenters. 10338 Or, of course, you could just use the exact type in your own code. 10339 10340 You return true if you handle the given url, or false if not. Just returning the result of [dispatcher] will do a 10341 good job. 10342 10343 10344 +/ 10345 auto dispatchTo(alias handler)(string urlPrefix) { 10346 assert(urlPrefix[0] == '/'); 10347 assert(urlPrefix[$-1] != '/'); 10348 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, const void* details) { 10349 return handler(DispatcherData!Presenter(cgi, presenter, urlPrefix.length)); 10350 } 10351 10352 return DispatcherDefinition!(internalHandler)(urlPrefix, false); 10353 } 10354 10355 /+ 10356 /++ 10357 See [serveStaticFile] if you want to serve a file off disk. 10358 +/ 10359 auto serveStaticData(string urlPrefix, const(void)[] data, string contentType) { 10360 10361 } 10362 +/ 10363 10364 /++ 10365 A URL dispatcher. 10366 10367 --- 10368 if(cgi.dispatcher!( 10369 "/api/".serveApi!MyApiClass, 10370 "/objects/lol".serveRestObject!MyRestObject, 10371 "/file.js".serveStaticFile, 10372 "/admin/".dispatchTo!adminHandler 10373 )) return; 10374 --- 10375 10376 10377 You define a series of url prefixes followed by handlers. 10378 10379 [dispatchTo] will send the request to another function for handling. 10380 You may want to do different pre- and post- processing there, for example, 10381 an authorization check and different page layout. You can use different 10382 presenters and different function chains. NOT IMPLEMENTED 10383 +/ 10384 template dispatcher(definitions...) { 10385 bool dispatcher(Presenter)(Cgi cgi, Presenter presenterArg = null) { 10386 static if(is(Presenter == typeof(null))) { 10387 static class GenericWebPresenter : WebPresenter!(GenericWebPresenter) {} 10388 auto presenter = new GenericWebPresenter(); 10389 } else 10390 alias presenter = presenterArg; 10391 10392 return dispatcher(DispatcherData!(typeof(presenter))(cgi, presenter, 0)); 10393 } 10394 10395 bool dispatcher(DispatcherData)(DispatcherData dispatcherData) if(!is(DispatcherData : Cgi)) { 10396 // I can prolly make this more efficient later but meh. 10397 foreach(definition; definitions) { 10398 if(definition.rejectFurther) { 10399 if(dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart .. $] == definition.urlPrefix) { 10400 auto ret = definition.handler( 10401 dispatcherData.cgi.pathInfo[0 .. dispatcherData.pathInfoStart + definition.urlPrefix.length], 10402 dispatcherData.cgi, dispatcherData.presenter, definition.details); 10403 if(ret) 10404 return true; 10405 } 10406 } else if( 10407 dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart .. $].startsWith(definition.urlPrefix) && 10408 // cgi.d dispatcher urls must be complete or have a /; 10409 // "foo" -> thing should NOT match "foobar", just "foo" or "foo/thing" 10410 (definition.urlPrefix[$-1] == '/' || (dispatcherData.pathInfoStart + definition.urlPrefix.length) == dispatcherData.cgi.pathInfo.length 10411 || dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart + definition.urlPrefix.length] == '/') 10412 ) { 10413 auto ret = definition.handler( 10414 dispatcherData.cgi.pathInfo[0 .. dispatcherData.pathInfoStart + definition.urlPrefix.length], 10415 dispatcherData.cgi, dispatcherData.presenter, definition.details); 10416 if(ret) 10417 return true; 10418 } 10419 } 10420 return false; 10421 } 10422 } 10423 10424 }); 10425 10426 private struct StackBuffer { 10427 char[1024] initial = void; 10428 char[] buffer; 10429 size_t position; 10430 10431 this(int a) { 10432 buffer = initial[]; 10433 position = 0; 10434 } 10435 10436 void add(in char[] what) { 10437 if(position + what.length > buffer.length) 10438 buffer.length = position + what.length + 1024; // reallocate with GC to handle special cases 10439 buffer[position .. position + what.length] = what[]; 10440 position += what.length; 10441 } 10442 10443 void add(in char[] w1, in char[] w2, in char[] w3 = null) { 10444 add(w1); 10445 add(w2); 10446 add(w3); 10447 } 10448 10449 void add(long v) { 10450 char[16] buffer = void; 10451 auto pos = buffer.length; 10452 bool negative; 10453 if(v < 0) { 10454 negative = true; 10455 v = -v; 10456 } 10457 do { 10458 buffer[--pos] = cast(char) (v % 10 + '0'); 10459 v /= 10; 10460 } while(v); 10461 10462 if(negative) 10463 buffer[--pos] = '-'; 10464 10465 auto res = buffer[pos .. $]; 10466 10467 add(res[]); 10468 } 10469 10470 char[] get() @nogc { 10471 return buffer[0 .. position]; 10472 } 10473 } 10474 10475 // duplicated in http2.d 10476 private static string getHttpCodeText(int code) pure nothrow @nogc { 10477 switch(code) { 10478 case 200: return "200 OK"; 10479 case 201: return "201 Created"; 10480 case 202: return "202 Accepted"; 10481 case 203: return "203 Non-Authoritative Information"; 10482 case 204: return "204 No Content"; 10483 case 205: return "205 Reset Content"; 10484 case 206: return "206 Partial Content"; 10485 // 10486 case 300: return "300 Multiple Choices"; 10487 case 301: return "301 Moved Permanently"; 10488 case 302: return "302 Found"; 10489 case 303: return "303 See Other"; 10490 case 304: return "304 Not Modified"; 10491 case 305: return "305 Use Proxy"; 10492 case 307: return "307 Temporary Redirect"; 10493 case 308: return "308 Permanent Redirect"; 10494 10495 // 10496 case 400: return "400 Bad Request"; 10497 case 401: return "401 Unauthorized"; 10498 case 402: return "402 Payment Required"; 10499 case 403: return "403 Forbidden"; 10500 case 404: return "404 Not Found"; 10501 case 405: return "405 Method Not Allowed"; 10502 case 406: return "406 Not Acceptable"; 10503 case 407: return "407 Proxy Authentication Required"; 10504 case 408: return "408 Request Timeout"; 10505 case 409: return "409 Conflict"; 10506 case 410: return "410 Gone"; 10507 case 411: return "411 Length Required"; 10508 case 412: return "412 Precondition Failed"; 10509 case 413: return "413 Payload Too Large"; 10510 case 414: return "414 URI Too Long"; 10511 case 415: return "415 Unsupported Media Type"; 10512 case 416: return "416 Range Not Satisfiable"; 10513 case 417: return "417 Expectation Failed"; 10514 case 418: return "418 I'm a teapot"; 10515 case 421: return "421 Misdirected Request"; 10516 case 422: return "422 Unprocessable Entity (WebDAV)"; 10517 case 423: return "423 Locked (WebDAV)"; 10518 case 424: return "424 Failed Dependency (WebDAV)"; 10519 case 425: return "425 Too Early"; 10520 case 426: return "426 Upgrade Required"; 10521 case 428: return "428 Precondition Required"; 10522 case 431: return "431 Request Header Fields Too Large"; 10523 case 451: return "451 Unavailable For Legal Reasons"; 10524 10525 case 500: return "500 Internal Server Error"; 10526 case 501: return "501 Not Implemented"; 10527 case 502: return "502 Bad Gateway"; 10528 case 503: return "503 Service Unavailable"; 10529 case 504: return "504 Gateway Timeout"; 10530 case 505: return "505 HTTP Version Not Supported"; 10531 case 506: return "506 Variant Also Negotiates"; 10532 case 507: return "507 Insufficient Storage (WebDAV)"; 10533 case 508: return "508 Loop Detected (WebDAV)"; 10534 case 510: return "510 Not Extended"; 10535 case 511: return "511 Network Authentication Required"; 10536 // 10537 default: assert(0, "Unsupported http code"); 10538 } 10539 } 10540 10541 10542 /+ 10543 /++ 10544 This is the beginnings of my web.d 2.0 - it dispatches web requests to a class object. 10545 10546 It relies on jsvar.d and dom.d. 10547 10548 10549 You can get javascript out of it to call. The generated functions need to look 10550 like 10551 10552 function name(a,b,c,d,e) { 10553 return _call("name", {"realName":a,"sds":b}); 10554 } 10555 10556 And _call returns an object you can call or set up or whatever. 10557 +/ 10558 bool apiDispatcher()(Cgi cgi) { 10559 import arsd.jsvar; 10560 import arsd.dom; 10561 } 10562 +/ 10563 /* 10564 Copyright: Adam D. Ruppe, 2008 - 2021 10565 License: [http://www.boost.org/LICENSE_1_0.txt|Boost License 1.0]. 10566 Authors: Adam D. Ruppe 10567 10568 Copyright Adam D. Ruppe 2008 - 2021. 10569 Distributed under the Boost Software License, Version 1.0. 10570 (See accompanying file LICENSE_1_0.txt or copy at 10571 http://www.boost.org/LICENSE_1_0.txt) 10572 */