1 // FIXME: if an exception is thrown, we shouldn't necessarily cache... 2 // FIXME: there's some annoying duplication of code in the various versioned mains 3 4 // add the Range header in there too. should return 206 5 6 // FIXME: cgi per-request arena allocator 7 8 // i need to add a bunch of type templates for validations... mayne @NotNull or NotNull! 9 10 // FIXME: I might make a cgi proxy class which can change things; the underlying one is still immutable 11 // but the later one can edit and simplify the api. You'd have to use the subclass tho! 12 13 /* 14 void foo(int f, @("test") string s) {} 15 16 void main() { 17 static if(is(typeof(foo) Params == __parameters)) 18 //pragma(msg, __traits(getAttributes, Params[0])); 19 pragma(msg, __traits(getAttributes, Params[1..2])); 20 else 21 pragma(msg, "fail"); 22 } 23 */ 24 25 // Note: spawn-fcgi can help with fastcgi on nginx 26 27 // FIXME: to do: add openssl optionally 28 // make sure embedded_httpd doesn't send two answers if one writes() then dies 29 30 // future direction: websocket as a separate process that you can sendfile to for an async passoff of those long-lived connections 31 32 /* 33 Session manager process: it spawns a new process, passing a 34 command line argument, to just be a little key/value store 35 of some serializable struct. On Windows, it CreateProcess. 36 On Linux, it can just fork or maybe fork/exec. The session 37 key is in a cookie. 38 39 Server-side event process: spawns an async manager. You can 40 push stuff out to channel ids and the clients listen to it. 41 42 websocket process: spawns an async handler. They can talk to 43 each other or get info from a cgi request. 44 45 Tempting to put web.d 2.0 in here. It would: 46 * map urls and form generation to functions 47 * have data presentation magic 48 * do the skeleton stuff like 1.0 49 * auto-cache generated stuff in files (at least if pure?) 50 * introspect functions in json for consumers 51 52 53 https://linux.die.net/man/3/posix_spawn 54 */ 55 56 /++ 57 Provides a uniform server-side API for CGI, FastCGI, SCGI, and HTTP web applications. 58 59 --- 60 import arsd.cgi; 61 62 // Instead of writing your own main(), you should write a function 63 // that takes a Cgi param, and use mixin GenericMain 64 // for maximum compatibility with different web servers. 65 void hello(Cgi cgi) { 66 cgi.setResponseContentType("text/plain"); 67 68 if("name" in cgi.get) 69 cgi.write("Hello, " ~ cgi.get["name"]); 70 else 71 cgi.write("Hello, world!"); 72 } 73 74 mixin GenericMain!hello; 75 --- 76 77 Test on console (works in any interface mode): 78 $(CONSOLE 79 $ ./cgi_hello GET / name=whatever 80 ) 81 82 If using http version (default on `dub` builds, or on custom builds when passing `-version=embedded_httpd` to dmd): 83 $(CONSOLE 84 $ ./cgi_hello --port 8080 85 # now you can go to http://localhost:8080/?name=whatever 86 ) 87 88 89 Compile_versions: 90 91 If you are using `dub`, use: 92 93 ```sdlang 94 subConfiguration "arsd-official:cgi" "VALUE_HERE" 95 ``` 96 97 or to dub.json: 98 99 ```json 100 "subConfigurations": {"arsd-official:cgi": "VALUE_HERE"} 101 ``` 102 103 to change versions. The possible options for `VALUE_HERE` are: 104 105 $(LIST 106 * `embedded_httpd` for the embedded httpd version (built-in web server). This is the default. 107 * `cgi` for traditional cgi binaries. 108 * `fastcgi` for FastCGI builds. 109 * `scgi` for SCGI builds. 110 * `stdio_http` for speaking raw http over stdin and stdout. See [RequestServer.serveSingleHttpConnectionOnStdio] for more information. 111 ) 112 113 With dmd, use: 114 115 $(TABLE_ROWS 116 117 * + Interfaces 118 + (mutually exclusive) 119 120 * - `-version=plain_cgi` 121 - The default building the module alone without dub - a traditional, plain CGI executable will be generated. 122 * - `-version=embedded_httpd` 123 - A HTTP server will be embedded in the generated executable. This is default when building with dub. 124 * - `-version=fastcgi` 125 - A FastCGI executable will be generated. 126 * - `-version=scgi` 127 - A SCGI (SimpleCGI) executable will be generated. 128 129 * - `-version=embedded_httpd_threads` 130 - The embedded HTTP server will use a single process with a thread pool. (use instead of plain `embedded_httpd` if you want this specific implementation) 131 * - `-version=embedded_httpd_processes` 132 - The embedded HTTP server will use a prefork style process pool. (use instead of plain `embedded_httpd` if you want this specific implementation) 133 * - `-version=embedded_httpd_processes_accept_after_fork` 134 - It will call accept() in each child process, after forking. This is currently the only option, though I am experimenting with other ideas. You probably should NOT specify this right now. 135 * - `-version=stdio_http` 136 - The embedded HTTP server will be spoken over stdin and stdout. 137 138 * + Tweaks 139 + (can be used together with others) 140 141 * - `-version=cgi_with_websocket` 142 - The CGI class has websocket server support. 143 144 * - `-version=with_openssl` 145 - not currently used 146 * - `-version=cgi_embedded_sessions` 147 - The session server will be embedded in the cgi.d server process 148 * - `-version=cgi_session_server_process` 149 - The session will be provided in a separate process, provided by cgi.d. 150 ) 151 152 Compile_and_run: 153 154 For CGI, `dmd yourfile.d cgi.d` then put the executable in your cgi-bin directory. 155 156 For FastCGI: `dmd yourfile.d cgi.d -version=fastcgi` and run it. spawn-fcgi helps on nginx. You can put the file in the directory for Apache. On IIS, run it with a port on the command line (this causes it to call FCGX_OpenSocket, which can work on nginx too). 157 158 For SCGI: `dmd yourfile.d cgi.d -version=scgi` and run the executable, providing a port number on the command line. 159 160 For an embedded HTTP server, run `dmd yourfile.d cgi.d -version=embedded_httpd` and run the generated program. It listens on port 8085 by default. You can change this on the command line with the --port option when running your program. 161 162 You can also simulate a request by passing parameters on the command line, like: 163 164 $(CONSOLE 165 ./yourprogram GET / name=adr 166 ) 167 168 And it will print the result to stdout. 169 170 CGI_Setup_tips: 171 172 On Apache, you may do `SetHandler cgi-script` in your `.htaccess` file. 173 174 Integration_tips: 175 176 cgi.d works well with dom.d for generating html. You may also use web.d for other utilities and automatic api wrapping. 177 178 dom.d usage: 179 180 --- 181 import arsd.cgi; 182 import arsd.dom; 183 184 void hello_dom(Cgi cgi) { 185 auto document = new Document(); 186 187 static import std.file; 188 // parse the file in strict mode, requiring it to be well-formed UTF-8 XHTML 189 // (You'll appreciate this if you've ever had to deal with a missing </div> 190 // or something in a php or erb template before that would randomly mess up 191 // the output in your browser. Just check it and throw an exception early!) 192 // 193 // You could also hard-code a template or load one at compile time with an 194 // import expression, but you might appreciate making it a regular file 195 // because that means it can be more easily edited by the frontend team and 196 // they can see their changes without needing to recompile the program. 197 // 198 // Note on CTFE: if you do choose to load a static file at compile time, 199 // you *can* parse it in CTFE using enum, which will cause it to throw at 200 // compile time, which is kinda cool too. Be careful in modifying that document, 201 // though, as it will be a static instance. You might want to clone on on demand, 202 // or perhaps modify it lazily as you print it out. (Try element.tree, it returns 203 // a range of elements which you could send through std.algorithm functions. But 204 // since my selector implementation doesn't work on that level yet, you'll find that 205 // harder to use. Of course, you could make a static list of matching elements and 206 // then use a simple e is e2 predicate... :) ) 207 document.parseUtf8(std.file.read("your_template.html"), true, true); 208 209 // fill in data using DOM functions, so placing it is in the hands of HTML 210 // and it will be properly encoded as text too. 211 // 212 // Plain html templates can't run server side logic, but I think that's a 213 // good thing - it keeps them simple. You may choose to extend the html, 214 // but I think it is best to try to stick to standard elements and fill them 215 // in with requested data with IDs or class names. A further benefit of 216 // this is the designer can also highlight data based on sources in the CSS. 217 // 218 // However, all of dom.d is available, so you can format your data however 219 // you like. You can do partial templates with innerHTML too, or perhaps better, 220 // injecting cloned nodes from a partial document. 221 // 222 // There's a lot of possibilities. 223 document["#name"].innerText = cgi.request("name", "default name"); 224 225 // send the document to the browser. The second argument to `cgi.write` 226 // indicates that this is all the data at once, enabling a few small 227 // optimizations. 228 cgi.write(document.toString(), true); 229 } 230 --- 231 232 Concepts: 233 Input: [Cgi.get], [Cgi.post], [Cgi.request], [Cgi.files], [Cgi.cookies], [Cgi.pathInfo], [Cgi.requestMethod], 234 and HTTP headers ([Cgi.headers], [Cgi.userAgent], [Cgi.referrer], [Cgi.accept], [Cgi.authorization], [Cgi.lastEventId]) 235 236 Output: [Cgi.write], [Cgi.header], [Cgi.setResponseStatus], [Cgi.setResponseContentType], [Cgi.gzipResponse] 237 238 Cookies: [Cgi.setCookie], [Cgi.clearCookie], [Cgi.cookie], [Cgi.cookies] 239 240 Caching: [Cgi.setResponseExpires], [Cgi.updateResponseExpires], [Cgi.setCache] 241 242 Redirections: [Cgi.setResponseLocation] 243 244 Other Information: [Cgi.remoteAddress], [Cgi.https], [Cgi.port], [Cgi.scriptName], [Cgi.requestUri], [Cgi.getCurrentCompleteUri], [Cgi.onRequestBodyDataReceived] 245 246 Overriding behavior: [Cgi.handleIncomingDataChunk], [Cgi.prepareForIncomingDataChunks], [Cgi.cleanUpPostDataState] 247 248 Installing: Apache, IIS, CGI, FastCGI, SCGI, embedded HTTPD (not recommended for production use) 249 250 Guide_for_PHP_users: 251 If you are coming from PHP, here's a quick guide to help you get started: 252 253 $(SIDE_BY_SIDE 254 $(COLUMN 255 ```php 256 <?php 257 $foo = $_POST["foo"]; 258 $bar = $_GET["bar"]; 259 $baz = $_COOKIE["baz"]; 260 261 $user_ip = $_SERVER["REMOTE_ADDR"]; 262 $host = $_SERVER["HTTP_HOST"]; 263 $path = $_SERVER["PATH_INFO"]; 264 265 setcookie("baz", "some value"); 266 267 echo "hello!"; 268 ?> 269 ``` 270 ) 271 $(COLUMN 272 --- 273 import arsd.cgi; 274 void app(Cgi cgi) { 275 string foo = cgi.post["foo"]; 276 string bar = cgi.get["bar"]; 277 string baz = cgi.cookies["baz"]; 278 279 string user_ip = cgi.remoteAddress; 280 string host = cgi.host; 281 string path = cgi.pathInfo; 282 283 cgi.setCookie("baz", "some value"); 284 285 cgi.write("hello!"); 286 } 287 288 mixin GenericMain!app 289 --- 290 ) 291 ) 292 293 $(H3 Array elements) 294 295 296 In PHP, you can give a form element a name like `"something[]"`, and then 297 `$_POST["something"]` gives an array. In D, you can use whatever name 298 you want, and access an array of values with the `cgi.getArray["name"]` and 299 `cgi.postArray["name"]` members. 300 301 $(H3 Databases) 302 303 PHP has a lot of stuff in its standard library. cgi.d doesn't include most 304 of these, but the rest of my arsd repository has much of it. For example, 305 to access a MySQL database, download `database.d` and `mysql.d` from my 306 github repo, and try this code (assuming, of course, your database is 307 set up): 308 309 --- 310 import arsd.cgi; 311 import arsd.mysql; 312 313 void app(Cgi cgi) { 314 auto database = new MySql("localhost", "username", "password", "database_name"); 315 foreach(row; mysql.query("SELECT count(id) FROM people")) 316 cgi.write(row[0] ~ " people in database"); 317 } 318 319 mixin GenericMain!app; 320 --- 321 322 Similar modules are available for PostgreSQL, Microsoft SQL Server, and SQLite databases, 323 implementing the same basic interface. 324 325 See_Also: 326 327 You may also want to see [arsd.dom], [arsd.web], and [arsd.html] for more code for making 328 web applications. 329 330 For working with json, try [arsd.jsvar]. 331 332 [arsd.database], [arsd.mysql], [arsd.postgres], [arsd.mssql], and [arsd.sqlite] can help in 333 accessing databases. 334 335 If you are looking to access a web application via HTTP, try [std.net.curl], [arsd.curl], or [arsd.http2]. 336 337 Copyright: 338 339 cgi.d copyright 2008-2021, Adam D. Ruppe. Provided under the Boost Software License. 340 341 Yes, this file is old, and yes, it is still actively maintained and used. 342 +/ 343 module arsd.cgi; 344 345 version(Demo) 346 unittest { 347 348 } 349 350 static import std.file; 351 352 // for a single thread, linear request thing, use: 353 // -version=embedded_httpd_threads -version=cgi_no_threads 354 355 version(Posix) { 356 version(CRuntime_Musl) { 357 358 } else version(minimal) { 359 360 } else { 361 version(GNU) { 362 // GDC doesn't support static foreach so I had to cheat on it :( 363 } else version(FreeBSD) { 364 // I never implemented the fancy stuff there either 365 } else { 366 version=with_breaking_cgi_features; 367 version=with_sendfd; 368 version=with_addon_servers; 369 } 370 } 371 } 372 373 void cloexec(int fd) { 374 version(Posix) { 375 import core.sys.posix.fcntl; 376 fcntl(fd, F_SETFD, FD_CLOEXEC); 377 } 378 } 379 380 void cloexec(Socket s) { 381 version(Posix) { 382 import core.sys.posix.fcntl; 383 fcntl(s.handle, F_SETFD, FD_CLOEXEC); 384 } 385 } 386 387 version(embedded_httpd_hybrid) { 388 version=embedded_httpd_threads; 389 version(cgi_no_fork) {} else 390 version=cgi_use_fork; 391 version=cgi_use_fiber; 392 } 393 394 // the servers must know about the connections to talk to them; the interfaces are vital 395 version(with_addon_servers) 396 version=with_addon_servers_connections; 397 398 version(embedded_httpd) { 399 version(linux) 400 version=embedded_httpd_processes; 401 else { 402 version=embedded_httpd_threads; 403 } 404 405 /* 406 version(with_openssl) { 407 pragma(lib, "crypto"); 408 pragma(lib, "ssl"); 409 } 410 */ 411 } 412 413 version(embedded_httpd_processes) 414 version=embedded_httpd_processes_accept_after_fork; // I am getting much better average performance on this, so just keeping it. But the other way MIGHT help keep the variation down so i wanna keep the code to play with later 415 416 version(embedded_httpd_threads) { 417 // unless the user overrides the default.. 418 version(cgi_session_server_process) 419 {} 420 else 421 version=cgi_embedded_sessions; 422 } 423 version(scgi) { 424 // unless the user overrides the default.. 425 version(cgi_session_server_process) 426 {} 427 else 428 version=cgi_embedded_sessions; 429 } 430 431 // fall back if the other is not defined so we can cleanly version it below 432 version(cgi_embedded_sessions) {} 433 else version=cgi_session_server_process; 434 435 436 version=cgi_with_websocket; 437 438 enum long defaultMaxContentLength = 5_000_000; 439 440 /* 441 442 To do a file download offer in the browser: 443 444 cgi.setResponseContentType("text/csv"); 445 cgi.header("Content-Disposition: attachment; filename=\"customers.csv\""); 446 */ 447 448 // FIXME: the location header is supposed to be an absolute url I guess. 449 450 // FIXME: would be cool to flush part of a dom document before complete 451 // somehow in here and dom.d. 452 453 454 // these are public so you can mixin GenericMain. 455 // FIXME: use a function level import instead! 456 public import std.string; 457 public import std.stdio; 458 public import std.conv; 459 import std.uri; 460 import std.uni; 461 import std.algorithm.comparison; 462 import std.algorithm.searching; 463 import std.exception; 464 import std.base64; 465 static import std.algorithm; 466 import std.datetime; 467 import std.range; 468 469 import std.process; 470 471 import std.zlib; 472 473 474 T[] consume(T)(T[] range, int count) { 475 if(count > range.length) 476 count = range.length; 477 return range[count..$]; 478 } 479 480 int locationOf(T)(T[] data, string item) { 481 const(ubyte[]) d = cast(const(ubyte[])) data; 482 const(ubyte[]) i = cast(const(ubyte[])) item; 483 484 // this is a vague sanity check to ensure we aren't getting insanely 485 // sized input that will infinite loop below. it should never happen; 486 // even huge file uploads ought to come in smaller individual pieces. 487 if(d.length > (int.max/2)) 488 throw new Exception("excessive block of input"); 489 490 for(int a = 0; a < d.length; a++) { 491 if(a + i.length > d.length) 492 return -1; 493 if(d[a..a+i.length] == i) 494 return a; 495 } 496 497 return -1; 498 } 499 500 /// If you are doing a custom cgi class, mixing this in can take care of 501 /// the required constructors for you 502 mixin template ForwardCgiConstructors() { 503 this(long maxContentLength = defaultMaxContentLength, 504 string[string] env = null, 505 const(ubyte)[] delegate() readdata = null, 506 void delegate(const(ubyte)[]) _rawDataOutput = null, 507 void delegate() _flush = null 508 ) { super(maxContentLength, env, readdata, _rawDataOutput, _flush); } 509 510 this(string[] args) { super(args); } 511 512 this( 513 BufferedInputRange inputData, 514 string address, ushort _port, 515 int pathInfoStarts = 0, 516 bool _https = false, 517 void delegate(const(ubyte)[]) _rawDataOutput = null, 518 void delegate() _flush = null, 519 // this pointer tells if the connection is supposed to be closed after we handle this 520 bool* closeConnection = null) 521 { 522 super(inputData, address, _port, pathInfoStarts, _https, _rawDataOutput, _flush, closeConnection); 523 } 524 525 this(BufferedInputRange ir, bool* closeConnection) { super(ir, closeConnection); } 526 } 527 528 /// thrown when a connection is closed remotely while we waiting on data from it 529 class ConnectionClosedException : Exception { 530 this(string message, string file = __FILE__, size_t line = __LINE__, Throwable next = null) { 531 super(message, file, line, next); 532 } 533 } 534 535 536 version(Windows) { 537 // FIXME: ugly hack to solve stdin exception problems on Windows: 538 // reading stdin results in StdioException (Bad file descriptor) 539 // this is probably due to http://d.puremagic.com/issues/show_bug.cgi?id=3425 540 private struct stdin { 541 struct ByChunk { // Replicates std.stdio.ByChunk 542 private: 543 ubyte[] chunk_; 544 public: 545 this(size_t size) 546 in { 547 assert(size, "size must be larger than 0"); 548 } 549 body { 550 chunk_ = new ubyte[](size); 551 popFront(); 552 } 553 554 @property bool empty() const { 555 return !std.stdio.stdin.isOpen || std.stdio.stdin.eof; // Ugly, but seems to do the job 556 } 557 @property nothrow ubyte[] front() { return chunk_; } 558 void popFront() { 559 enforce(!empty, "Cannot call popFront on empty range"); 560 chunk_ = stdin.rawRead(chunk_); 561 } 562 } 563 564 import core.sys.windows.windows; 565 static: 566 567 static this() { 568 // Set stdin to binary mode 569 version(Win64) 570 _setmode(std.stdio.stdin.fileno(), 0x8000); 571 else 572 setmode(std.stdio.stdin.fileno(), 0x8000); 573 } 574 575 T[] rawRead(T)(T[] buf) { 576 uint bytesRead; 577 auto result = ReadFile(GetStdHandle(STD_INPUT_HANDLE), buf.ptr, cast(int) (buf.length * T.sizeof), &bytesRead, null); 578 579 if (!result) { 580 auto err = GetLastError(); 581 if (err == 38/*ERROR_HANDLE_EOF*/ || err == 109/*ERROR_BROKEN_PIPE*/) // 'good' errors meaning end of input 582 return buf[0..0]; 583 // Some other error, throw it 584 585 char* buffer; 586 scope(exit) LocalFree(buffer); 587 588 // FORMAT_MESSAGE_ALLOCATE_BUFFER = 0x00000100 589 // FORMAT_MESSAGE_FROM_SYSTEM = 0x00001000 590 FormatMessageA(0x1100, null, err, 0, cast(char*)&buffer, 256, null); 591 throw new Exception(to!string(buffer)); 592 } 593 enforce(!(bytesRead % T.sizeof), "I/O error"); 594 return buf[0..bytesRead / T.sizeof]; 595 } 596 597 auto byChunk(size_t sz) { return ByChunk(sz); } 598 599 void close() { 600 std.stdio.stdin.close; 601 } 602 } 603 } 604 605 /// The main interface with the web request 606 class Cgi { 607 public: 608 /// the methods a request can be 609 enum RequestMethod { GET, HEAD, POST, PUT, DELETE, // GET and POST are the ones that really work 610 // these are defined in the standard, but idk if they are useful for anything 611 OPTIONS, TRACE, CONNECT, 612 // These seem new, I have only recently seen them 613 PATCH, MERGE, 614 // this is an extension for when the method is not specified and you want to assume 615 CommandLine } 616 617 618 /+ 619 /++ 620 Cgi provides a per-request memory pool 621 622 +/ 623 void[] allocateMemory(size_t nBytes) { 624 625 } 626 627 /// ditto 628 void[] reallocateMemory(void[] old, size_t nBytes) { 629 630 } 631 632 /// ditto 633 void freeMemory(void[] memory) { 634 635 } 636 +/ 637 638 639 /* 640 import core.runtime; 641 auto args = Runtime.args(); 642 643 we can call the app a few ways: 644 645 1) set up the environment variables and call the app (manually simulating CGI) 646 2) simulate a call automatically: 647 ./app method 'uri' 648 649 for example: 650 ./app get /path?arg arg2=something 651 652 Anything on the uri is treated as query string etc 653 654 on get method, further args are appended to the query string (encoded automatically) 655 on post method, further args are done as post 656 657 658 @name means import from file "name". if name == -, it uses stdin 659 (so info=@- means set info to the value of stdin) 660 661 662 Other arguments include: 663 --cookie name=value (these are all concated together) 664 --header 'X-Something: cool' 665 --referrer 'something' 666 --port 80 667 --remote-address some.ip.address.here 668 --https yes 669 --user-agent 'something' 670 --userpass 'user:pass' 671 --authorization 'Basic base64encoded_user:pass' 672 --accept 'content' // FIXME: better example 673 --last-event-id 'something' 674 --host 'something.com' 675 676 Non-simulation arguments: 677 --port xxx listening port for non-cgi things (valid for the cgi interfaces) 678 --listening-host the ip address the application should listen on, or if you want to use unix domain sockets, it is here you can set them: `--listening-host unix:filename` or, on Linux, `--listening-host abstract:name`. 679 680 */ 681 682 /** Initializes it with command line arguments (for easy testing) */ 683 this(string[] args, void delegate(const(ubyte)[]) _rawDataOutput = null) { 684 rawDataOutput = _rawDataOutput; 685 // these are all set locally so the loop works 686 // without triggering errors in dmd 2.064 687 // we go ahead and set them at the end of it to the this version 688 int port; 689 string referrer; 690 string remoteAddress; 691 string userAgent; 692 string authorization; 693 string origin; 694 string accept; 695 string lastEventId; 696 bool https; 697 string host; 698 RequestMethod requestMethod; 699 string requestUri; 700 string pathInfo; 701 string queryString; 702 703 bool lookingForMethod; 704 bool lookingForUri; 705 string nextArgIs; 706 707 string _cookie; 708 string _queryString; 709 string[][string] _post; 710 string[string] _headers; 711 712 string[] breakUp(string s) { 713 string k, v; 714 auto idx = s.indexOf("="); 715 if(idx == -1) { 716 k = s; 717 } else { 718 k = s[0 .. idx]; 719 v = s[idx + 1 .. $]; 720 } 721 722 return [k, v]; 723 } 724 725 lookingForMethod = true; 726 727 scriptName = args[0]; 728 scriptFileName = args[0]; 729 730 environmentVariables = cast(const) environment.toAA; 731 732 foreach(arg; args[1 .. $]) { 733 if(arg.startsWith("--")) { 734 nextArgIs = arg[2 .. $]; 735 } else if(nextArgIs.length) { 736 if (nextArgIs == "cookie") { 737 auto info = breakUp(arg); 738 if(_cookie.length) 739 _cookie ~= "; "; 740 _cookie ~= std.uri.encodeComponent(info[0]) ~ "=" ~ std.uri.encodeComponent(info[1]); 741 } 742 else if (nextArgIs == "port") { 743 port = to!int(arg); 744 } 745 else if (nextArgIs == "referrer") { 746 referrer = arg; 747 } 748 else if (nextArgIs == "remote-address") { 749 remoteAddress = arg; 750 } 751 else if (nextArgIs == "user-agent") { 752 userAgent = arg; 753 } 754 else if (nextArgIs == "authorization") { 755 authorization = arg; 756 } 757 else if (nextArgIs == "userpass") { 758 authorization = "Basic " ~ Base64.encode(cast(immutable(ubyte)[]) (arg)).idup; 759 } 760 else if (nextArgIs == "origin") { 761 origin = arg; 762 } 763 else if (nextArgIs == "accept") { 764 accept = arg; 765 } 766 else if (nextArgIs == "last-event-id") { 767 lastEventId = arg; 768 } 769 else if (nextArgIs == "https") { 770 if(arg == "yes") 771 https = true; 772 } 773 else if (nextArgIs == "header") { 774 string thing, other; 775 auto idx = arg.indexOf(":"); 776 if(idx == -1) 777 throw new Exception("need a colon in a http header"); 778 thing = arg[0 .. idx]; 779 other = arg[idx + 1.. $]; 780 _headers[thing.strip.toLower()] = other.strip; 781 } 782 else if (nextArgIs == "host") { 783 host = arg; 784 } 785 // else 786 // skip, we don't know it but that's ok, it might be used elsewhere so no error 787 788 nextArgIs = null; 789 } else if(lookingForMethod) { 790 lookingForMethod = false; 791 lookingForUri = true; 792 793 if(arg.asLowerCase().equal("commandline")) 794 requestMethod = RequestMethod.CommandLine; 795 else 796 requestMethod = to!RequestMethod(arg.toUpper()); 797 } else if(lookingForUri) { 798 lookingForUri = false; 799 800 requestUri = arg; 801 802 auto idx = arg.indexOf("?"); 803 if(idx == -1) 804 pathInfo = arg; 805 else { 806 pathInfo = arg[0 .. idx]; 807 _queryString = arg[idx + 1 .. $]; 808 } 809 } else { 810 // it is an argument of some sort 811 if(requestMethod == Cgi.RequestMethod.POST || requestMethod == Cgi.RequestMethod.PATCH || requestMethod == Cgi.RequestMethod.PUT) { 812 auto parts = breakUp(arg); 813 _post[parts[0]] ~= parts[1]; 814 allPostNamesInOrder ~= parts[0]; 815 allPostValuesInOrder ~= parts[1]; 816 } else { 817 if(_queryString.length) 818 _queryString ~= "&"; 819 auto parts = breakUp(arg); 820 _queryString ~= std.uri.encodeComponent(parts[0]) ~ "=" ~ std.uri.encodeComponent(parts[1]); 821 } 822 } 823 } 824 825 acceptsGzip = false; 826 keepAliveRequested = false; 827 requestHeaders = cast(immutable) _headers; 828 829 cookie = _cookie; 830 cookiesArray = getCookieArray(); 831 cookies = keepLastOf(cookiesArray); 832 833 queryString = _queryString; 834 getArray = cast(immutable) decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 835 get = keepLastOf(getArray); 836 837 postArray = cast(immutable) _post; 838 post = keepLastOf(_post); 839 840 // FIXME 841 filesArray = null; 842 files = null; 843 844 isCalledWithCommandLineArguments = true; 845 846 this.port = port; 847 this.referrer = referrer; 848 this.remoteAddress = remoteAddress; 849 this.userAgent = userAgent; 850 this.authorization = authorization; 851 this.origin = origin; 852 this.accept = accept; 853 this.lastEventId = lastEventId; 854 this.https = https; 855 this.host = host; 856 this.requestMethod = requestMethod; 857 this.requestUri = requestUri; 858 this.pathInfo = pathInfo; 859 this.queryString = queryString; 860 this.postBody = null; 861 } 862 863 private { 864 string[] allPostNamesInOrder; 865 string[] allPostValuesInOrder; 866 string[] allGetNamesInOrder; 867 string[] allGetValuesInOrder; 868 } 869 870 CgiConnectionHandle getOutputFileHandle() { 871 return _outputFileHandle; 872 } 873 874 CgiConnectionHandle _outputFileHandle = INVALID_CGI_CONNECTION_HANDLE; 875 876 /** Initializes it using a CGI or CGI-like interface */ 877 this(long maxContentLength = defaultMaxContentLength, 878 // use this to override the environment variable listing 879 in string[string] env = null, 880 // and this should return a chunk of data. return empty when done 881 const(ubyte)[] delegate() readdata = null, 882 // finally, use this to do custom output if needed 883 void delegate(const(ubyte)[]) _rawDataOutput = null, 884 // to flush teh custom output 885 void delegate() _flush = null 886 ) 887 { 888 889 // these are all set locally so the loop works 890 // without triggering errors in dmd 2.064 891 // we go ahead and set them at the end of it to the this version 892 int port; 893 string referrer; 894 string remoteAddress; 895 string userAgent; 896 string authorization; 897 string origin; 898 string accept; 899 string lastEventId; 900 bool https; 901 string host; 902 RequestMethod requestMethod; 903 string requestUri; 904 string pathInfo; 905 string queryString; 906 907 908 909 isCalledWithCommandLineArguments = false; 910 rawDataOutput = _rawDataOutput; 911 flushDelegate = _flush; 912 auto getenv = delegate string(string var) { 913 if(env is null) 914 return std.process.environment.get(var); 915 auto e = var in env; 916 if(e is null) 917 return null; 918 return *e; 919 }; 920 921 environmentVariables = env is null ? 922 cast(const) environment.toAA : 923 env; 924 925 // fetching all the request headers 926 string[string] requestHeadersHere; 927 foreach(k, v; env is null ? cast(const) environment.toAA() : env) { 928 if(k.startsWith("HTTP_")) { 929 requestHeadersHere[replace(k["HTTP_".length .. $].toLower(), "_", "-")] = v; 930 } 931 } 932 933 this.requestHeaders = assumeUnique(requestHeadersHere); 934 935 requestUri = getenv("REQUEST_URI"); 936 937 cookie = getenv("HTTP_COOKIE"); 938 cookiesArray = getCookieArray(); 939 cookies = keepLastOf(cookiesArray); 940 941 referrer = getenv("HTTP_REFERER"); 942 userAgent = getenv("HTTP_USER_AGENT"); 943 remoteAddress = getenv("REMOTE_ADDR"); 944 host = getenv("HTTP_HOST"); 945 pathInfo = getenv("PATH_INFO"); 946 947 queryString = getenv("QUERY_STRING"); 948 scriptName = getenv("SCRIPT_NAME"); 949 { 950 import core.runtime; 951 auto sfn = getenv("SCRIPT_FILENAME"); 952 scriptFileName = sfn.length ? sfn : Runtime.args[0]; 953 } 954 955 bool iis = false; 956 957 // Because IIS doesn't pass requestUri, we simulate it here if it's empty. 958 if(requestUri.length == 0) { 959 // IIS sometimes includes the script name as part of the path info - we don't want that 960 if(pathInfo.length >= scriptName.length && (pathInfo[0 .. scriptName.length] == scriptName)) 961 pathInfo = pathInfo[scriptName.length .. $]; 962 963 requestUri = scriptName ~ pathInfo ~ (queryString.length ? ("?" ~ queryString) : ""); 964 965 iis = true; // FIXME HACK - used in byChunk below - see bugzilla 6339 966 967 // FIXME: this works for apache and iis... but what about others? 968 } 969 970 971 auto ugh = decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 972 getArray = assumeUnique(ugh); 973 get = keepLastOf(getArray); 974 975 976 // NOTE: on shitpache, you need to specifically forward this 977 authorization = getenv("HTTP_AUTHORIZATION"); 978 // this is a hack because Apache is a shitload of fuck and 979 // refuses to send the real header to us. Compatible 980 // programs should send both the standard and X- versions 981 982 // NOTE: if you have access to .htaccess or httpd.conf, you can make this 983 // unnecessary with mod_rewrite, so it is commented 984 985 //if(authorization.length == 0) // if the std is there, use it 986 // authorization = getenv("HTTP_X_AUTHORIZATION"); 987 988 // the REDIRECT_HTTPS check is here because with an Apache hack, the port can become wrong 989 if(getenv("SERVER_PORT").length && getenv("REDIRECT_HTTPS") != "on") 990 port = to!int(getenv("SERVER_PORT")); 991 else 992 port = 0; // this was probably called from the command line 993 994 auto ae = getenv("HTTP_ACCEPT_ENCODING"); 995 if(ae.length && ae.indexOf("gzip") != -1) 996 acceptsGzip = true; 997 998 accept = getenv("HTTP_ACCEPT"); 999 lastEventId = getenv("HTTP_LAST_EVENT_ID"); 1000 1001 auto ka = getenv("HTTP_CONNECTION"); 1002 if(ka.length && ka.asLowerCase().canFind("keep-alive")) 1003 keepAliveRequested = true; 1004 1005 auto or = getenv("HTTP_ORIGIN"); 1006 origin = or; 1007 1008 auto rm = getenv("REQUEST_METHOD"); 1009 if(rm.length) 1010 requestMethod = to!RequestMethod(getenv("REQUEST_METHOD")); 1011 else 1012 requestMethod = RequestMethod.CommandLine; 1013 1014 // FIXME: hack on REDIRECT_HTTPS; this is there because the work app uses mod_rewrite which loses the https flag! So I set it with [E=HTTPS=%HTTPS] or whatever but then it gets translated to here so i want it to still work. This is arguably wrong but meh. 1015 https = (getenv("HTTPS") == "on" || getenv("REDIRECT_HTTPS") == "on"); 1016 1017 // FIXME: DOCUMENT_ROOT? 1018 1019 // FIXME: what about PUT? 1020 if(requestMethod == RequestMethod.POST || requestMethod == Cgi.RequestMethod.PATCH || requestMethod == Cgi.RequestMethod.PUT) { 1021 version(preserveData) // a hack to make forwarding simpler 1022 immutable(ubyte)[] data; 1023 size_t amountReceived = 0; 1024 auto contentType = getenv("CONTENT_TYPE"); 1025 1026 // FIXME: is this ever not going to be set? I guess it depends 1027 // on if the server de-chunks and buffers... seems like it has potential 1028 // to be slow if they did that. The spec says it is always there though. 1029 // And it has worked reliably for me all year in the live environment, 1030 // but some servers might be different. 1031 auto contentLength = to!size_t(getenv("CONTENT_LENGTH")); 1032 1033 immutable originalContentLength = contentLength; 1034 if(contentLength) { 1035 if(maxContentLength > 0 && contentLength > maxContentLength) { 1036 setResponseStatus("413 Request entity too large"); 1037 write("You tried to upload a file that is too large."); 1038 close(); 1039 throw new Exception("POST too large"); 1040 } 1041 prepareForIncomingDataChunks(contentType, contentLength); 1042 1043 1044 int processChunk(in ubyte[] chunk) { 1045 if(chunk.length > contentLength) { 1046 handleIncomingDataChunk(chunk[0..contentLength]); 1047 amountReceived += contentLength; 1048 contentLength = 0; 1049 return 1; 1050 } else { 1051 handleIncomingDataChunk(chunk); 1052 contentLength -= chunk.length; 1053 amountReceived += chunk.length; 1054 } 1055 if(contentLength == 0) 1056 return 1; 1057 1058 onRequestBodyDataReceived(amountReceived, originalContentLength); 1059 return 0; 1060 } 1061 1062 1063 if(readdata is null) { 1064 foreach(ubyte[] chunk; stdin.byChunk(iis ? contentLength : 4096)) 1065 if(processChunk(chunk)) 1066 break; 1067 } else { 1068 // we have a custom data source.. 1069 auto chunk = readdata(); 1070 while(chunk.length) { 1071 if(processChunk(chunk)) 1072 break; 1073 chunk = readdata(); 1074 } 1075 } 1076 1077 onRequestBodyDataReceived(amountReceived, originalContentLength); 1078 postArray = assumeUnique(pps._post); 1079 filesArray = assumeUnique(pps._files); 1080 files = keepLastOf(filesArray); 1081 post = keepLastOf(postArray); 1082 this.postBody = pps.postBody; 1083 cleanUpPostDataState(); 1084 } 1085 1086 version(preserveData) 1087 originalPostData = data; 1088 } 1089 // fixme: remote_user script name 1090 1091 1092 this.port = port; 1093 this.referrer = referrer; 1094 this.remoteAddress = remoteAddress; 1095 this.userAgent = userAgent; 1096 this.authorization = authorization; 1097 this.origin = origin; 1098 this.accept = accept; 1099 this.lastEventId = lastEventId; 1100 this.https = https; 1101 this.host = host; 1102 this.requestMethod = requestMethod; 1103 this.requestUri = requestUri; 1104 this.pathInfo = pathInfo; 1105 this.queryString = queryString; 1106 } 1107 1108 /// Cleans up any temporary files. Do not use the object 1109 /// after calling this. 1110 /// 1111 /// NOTE: it is called automatically by GenericMain 1112 // FIXME: this should be called if the constructor fails too, if it has created some garbage... 1113 void dispose() { 1114 foreach(file; files) { 1115 if(!file.contentInMemory) 1116 if(std.file.exists(file.contentFilename)) 1117 std.file.remove(file.contentFilename); 1118 } 1119 } 1120 1121 private { 1122 struct PostParserState { 1123 string contentType; 1124 string boundary; 1125 string localBoundary; // the ones used at the end or something lol 1126 bool isMultipart; 1127 bool needsSavedBody; 1128 1129 ulong expectedLength; 1130 ulong contentConsumed; 1131 immutable(ubyte)[] buffer; 1132 1133 // multipart parsing state 1134 int whatDoWeWant; 1135 bool weHaveAPart; 1136 string[] thisOnesHeaders; 1137 immutable(ubyte)[] thisOnesData; 1138 1139 string postBody; 1140 1141 UploadedFile piece; 1142 bool isFile = false; 1143 1144 size_t memoryCommitted; 1145 1146 // do NOT keep mutable references to these anywhere! 1147 // I assume they are unique in the constructor once we're all done getting data. 1148 string[][string] _post; 1149 UploadedFile[][string] _files; 1150 } 1151 1152 PostParserState pps; 1153 } 1154 1155 /// This represents a file the user uploaded via a POST request. 1156 static struct UploadedFile { 1157 /// If you want to create one of these structs for yourself from some data, 1158 /// use this function. 1159 static UploadedFile fromData(immutable(void)[] data, string name = null) { 1160 Cgi.UploadedFile f; 1161 f.filename = name; 1162 f.content = cast(immutable(ubyte)[]) data; 1163 f.contentInMemory = true; 1164 return f; 1165 } 1166 1167 string name; /// The name of the form element. 1168 string filename; /// The filename the user set. 1169 string contentType; /// The MIME type the user's browser reported. (Not reliable.) 1170 1171 /** 1172 For small files, cgi.d will buffer the uploaded file in memory, and make it 1173 directly accessible to you through the content member. I find this very convenient 1174 and somewhat efficient, since it can avoid hitting the disk entirely. (I 1175 often want to inspect and modify the file anyway!) 1176 1177 I find the file is very large, it is undesirable to eat that much memory just 1178 for a file buffer. In those cases, if you pass a large enough value for maxContentLength 1179 to the constructor so they are accepted, cgi.d will write the content to a temporary 1180 file that you can re-read later. 1181 1182 You can override this behavior by subclassing Cgi and overriding the protected 1183 handlePostChunk method. Note that the object is not initialized when you 1184 write that method - the http headers are available, but the cgi.post method 1185 is not. You may parse the file as it streams in using this method. 1186 1187 1188 Anyway, if the file is small enough to be in memory, contentInMemory will be 1189 set to true, and the content is available in the content member. 1190 1191 If not, contentInMemory will be set to false, and the content saved in a file, 1192 whose name will be available in the contentFilename member. 1193 1194 1195 Tip: if you know you are always dealing with small files, and want the convenience 1196 of ignoring this member, construct Cgi with a small maxContentLength. Then, if 1197 a large file comes in, it simply throws an exception (and HTTP error response) 1198 instead of trying to handle it. 1199 1200 The default value of maxContentLength in the constructor is for small files. 1201 */ 1202 bool contentInMemory = true; // the default ought to always be true 1203 immutable(ubyte)[] content; /// The actual content of the file, if contentInMemory == true 1204 string contentFilename; /// the file where we dumped the content, if contentInMemory == false. Note that if you want to keep it, you MUST move the file, since otherwise it is considered garbage when cgi is disposed. 1205 1206 /// 1207 ulong fileSize() { 1208 if(contentInMemory) 1209 return content.length; 1210 import std.file; 1211 return std.file.getSize(contentFilename); 1212 1213 } 1214 1215 /// 1216 void writeToFile(string filenameToSaveTo) const { 1217 import std.file; 1218 if(contentInMemory) 1219 std.file.write(filenameToSaveTo, content); 1220 else 1221 std.file.rename(contentFilename, filenameToSaveTo); 1222 } 1223 } 1224 1225 // given a content type and length, decide what we're going to do with the data.. 1226 protected void prepareForIncomingDataChunks(string contentType, ulong contentLength) { 1227 pps.expectedLength = contentLength; 1228 1229 auto terminator = contentType.indexOf(";"); 1230 if(terminator == -1) 1231 terminator = contentType.length; 1232 1233 pps.contentType = contentType[0 .. terminator]; 1234 auto b = contentType[terminator .. $]; 1235 if(b.length) { 1236 auto idx = b.indexOf("boundary="); 1237 if(idx != -1) { 1238 pps.boundary = b[idx + "boundary=".length .. $]; 1239 pps.localBoundary = "\r\n--" ~ pps.boundary; 1240 } 1241 } 1242 1243 // while a content type SHOULD be sent according to the RFC, it is 1244 // not required. We're told we SHOULD guess by looking at the content 1245 // but it seems to me that this only happens when it is urlencoded. 1246 if(pps.contentType == "application/x-www-form-urlencoded" || pps.contentType == "") { 1247 pps.isMultipart = false; 1248 pps.needsSavedBody = false; 1249 } else if(pps.contentType == "multipart/form-data") { 1250 pps.isMultipart = true; 1251 enforce(pps.boundary.length, "no boundary"); 1252 } else if(pps.contentType == "text/xml") { // FIXME: could this be special and load the post params 1253 // save the body so the application can handle it 1254 pps.isMultipart = false; 1255 pps.needsSavedBody = true; 1256 } else if(pps.contentType == "application/json") { // FIXME: this could prolly try to load post params too 1257 // save the body so the application can handle it 1258 pps.needsSavedBody = true; 1259 pps.isMultipart = false; 1260 } else { 1261 // the rest is 100% handled by the application. just save the body and send it to them 1262 pps.needsSavedBody = true; 1263 pps.isMultipart = false; 1264 } 1265 } 1266 1267 // handles streaming POST data. If you handle some other content type, you should 1268 // override this. If the data isn't the content type you want, you ought to call 1269 // super.handleIncomingDataChunk so regular forms and files still work. 1270 1271 // FIXME: I do some copying in here that I'm pretty sure is unnecessary, and the 1272 // file stuff I'm sure is inefficient. But, my guess is the real bottleneck is network 1273 // input anyway, so I'm not going to get too worked up about it right now. 1274 protected void handleIncomingDataChunk(const(ubyte)[] chunk) { 1275 if(chunk.length == 0) 1276 return; 1277 assert(chunk.length <= 32 * 1024 * 1024); // we use chunk size as a memory constraint thing, so 1278 // if we're passed big chunks, it might throw unnecessarily. 1279 // just pass it smaller chunks at a time. 1280 if(pps.isMultipart) { 1281 // multipart/form-data 1282 1283 1284 // FIXME: this might want to be factored out and factorized 1285 // need to make sure the stream hooks actually work. 1286 void pieceHasNewContent() { 1287 // we just grew the piece's buffer. Do we have to switch to file backing? 1288 if(pps.piece.contentInMemory) { 1289 if(pps.piece.content.length <= 10 * 1024 * 1024) 1290 // meh, I'm ok with it. 1291 return; 1292 else { 1293 // this is too big. 1294 if(!pps.isFile) 1295 throw new Exception("Request entity too large"); // a variable this big is kinda ridiculous, just reject it. 1296 else { 1297 // a file this large is probably acceptable though... let's use a backing file. 1298 pps.piece.contentInMemory = false; 1299 // FIXME: say... how do we intend to delete these things? cgi.dispose perhaps. 1300 1301 int count = 0; 1302 pps.piece.contentFilename = getTempDirectory() ~ "arsd_cgi_uploaded_file_" ~ to!string(getUtcTime()) ~ "-" ~ to!string(count); 1303 // odds are this loop will never be entered, but we want it just in case. 1304 while(std.file.exists(pps.piece.contentFilename)) { 1305 count++; 1306 pps.piece.contentFilename = getTempDirectory() ~ "arsd_cgi_uploaded_file_" ~ to!string(getUtcTime()) ~ "-" ~ to!string(count); 1307 } 1308 // I hope this creates the file pretty quickly, or the loop might be useless... 1309 // FIXME: maybe I should write some kind of custom transaction here. 1310 std.file.write(pps.piece.contentFilename, pps.piece.content); 1311 1312 pps.piece.content = null; 1313 } 1314 } 1315 } else { 1316 // it's already in a file, so just append it to what we have 1317 if(pps.piece.content.length) { 1318 // FIXME: this is surely very inefficient... we'll be calling this by 4kb chunk... 1319 std.file.append(pps.piece.contentFilename, pps.piece.content); 1320 pps.piece.content = null; 1321 } 1322 } 1323 } 1324 1325 1326 void commitPart() { 1327 if(!pps.weHaveAPart) 1328 return; 1329 1330 pieceHasNewContent(); // be sure the new content is handled every time 1331 1332 if(pps.isFile) { 1333 // I'm not sure if other environments put files in post or not... 1334 // I used to not do it, but I think I should, since it is there... 1335 pps._post[pps.piece.name] ~= pps.piece.filename; 1336 pps._files[pps.piece.name] ~= pps.piece; 1337 1338 allPostNamesInOrder ~= pps.piece.name; 1339 allPostValuesInOrder ~= pps.piece.filename; 1340 } else { 1341 pps._post[pps.piece.name] ~= cast(string) pps.piece.content; 1342 1343 allPostNamesInOrder ~= pps.piece.name; 1344 allPostValuesInOrder ~= cast(string) pps.piece.content; 1345 } 1346 1347 /* 1348 stderr.writeln("RECEIVED: ", pps.piece.name, "=", 1349 pps.piece.content.length < 1000 1350 ? 1351 to!string(pps.piece.content) 1352 : 1353 "too long"); 1354 */ 1355 1356 // FIXME: the limit here 1357 pps.memoryCommitted += pps.piece.content.length; 1358 1359 pps.weHaveAPart = false; 1360 pps.whatDoWeWant = 1; 1361 pps.thisOnesHeaders = null; 1362 pps.thisOnesData = null; 1363 1364 pps.piece = UploadedFile.init; 1365 pps.isFile = false; 1366 } 1367 1368 void acceptChunk() { 1369 pps.buffer ~= chunk; 1370 chunk = null; // we've consumed it into the buffer, so keeping it just brings confusion 1371 } 1372 1373 immutable(ubyte)[] consume(size_t howMuch) { 1374 pps.contentConsumed += howMuch; 1375 auto ret = pps.buffer[0 .. howMuch]; 1376 pps.buffer = pps.buffer[howMuch .. $]; 1377 return ret; 1378 } 1379 1380 dataConsumptionLoop: do { 1381 switch(pps.whatDoWeWant) { 1382 default: assert(0); 1383 case 0: 1384 acceptChunk(); 1385 // the format begins with two extra leading dashes, then we should be at the boundary 1386 if(pps.buffer.length < 2) 1387 return; 1388 assert(pps.buffer[0] == '-', "no leading dash"); 1389 consume(1); 1390 assert(pps.buffer[0] == '-', "no second leading dash"); 1391 consume(1); 1392 1393 pps.whatDoWeWant = 1; 1394 goto case 1; 1395 /* fallthrough */ 1396 case 1: // looking for headers 1397 // here, we should be lined up right at the boundary, which is followed by a \r\n 1398 1399 // want to keep the buffer under control in case we're under attack 1400 //stderr.writeln("here once"); 1401 //if(pps.buffer.length + chunk.length > 70 * 1024) // they should be < 1 kb really.... 1402 // throw new Exception("wtf is up with the huge mime part headers"); 1403 1404 acceptChunk(); 1405 1406 if(pps.buffer.length < pps.boundary.length) 1407 return; // not enough data, since there should always be a boundary here at least 1408 1409 if(pps.contentConsumed + pps.boundary.length + 6 == pps.expectedLength) { 1410 assert(pps.buffer.length == pps.boundary.length + 4 + 2); // --, --, and \r\n 1411 // we *should* be at the end here! 1412 assert(pps.buffer[0] == '-'); 1413 consume(1); 1414 assert(pps.buffer[0] == '-'); 1415 consume(1); 1416 1417 // the message is terminated by --BOUNDARY--\r\n (after a \r\n leading to the boundary) 1418 assert(pps.buffer[0 .. pps.boundary.length] == cast(const(ubyte[])) pps.boundary, 1419 "not lined up on boundary " ~ pps.boundary); 1420 consume(pps.boundary.length); 1421 1422 assert(pps.buffer[0] == '-'); 1423 consume(1); 1424 assert(pps.buffer[0] == '-'); 1425 consume(1); 1426 1427 assert(pps.buffer[0] == '\r'); 1428 consume(1); 1429 assert(pps.buffer[0] == '\n'); 1430 consume(1); 1431 1432 assert(pps.buffer.length == 0); 1433 assert(pps.contentConsumed == pps.expectedLength); 1434 break dataConsumptionLoop; // we're done! 1435 } else { 1436 // we're not done yet. We should be lined up on a boundary. 1437 1438 // But, we want to ensure the headers are here before we consume anything! 1439 auto headerEndLocation = locationOf(pps.buffer, "\r\n\r\n"); 1440 if(headerEndLocation == -1) 1441 return; // they *should* all be here, so we can handle them all at once. 1442 1443 assert(pps.buffer[0 .. pps.boundary.length] == cast(const(ubyte[])) pps.boundary, 1444 "not lined up on boundary " ~ pps.boundary); 1445 1446 consume(pps.boundary.length); 1447 // the boundary is always followed by a \r\n 1448 assert(pps.buffer[0] == '\r'); 1449 consume(1); 1450 assert(pps.buffer[0] == '\n'); 1451 consume(1); 1452 } 1453 1454 // re-running since by consuming the boundary, we invalidate the old index. 1455 auto headerEndLocation = locationOf(pps.buffer, "\r\n\r\n"); 1456 assert(headerEndLocation >= 0, "no header"); 1457 auto thisOnesHeaders = pps.buffer[0..headerEndLocation]; 1458 1459 consume(headerEndLocation + 4); // The +4 is the \r\n\r\n that caps it off 1460 1461 pps.thisOnesHeaders = split(cast(string) thisOnesHeaders, "\r\n"); 1462 1463 // now we'll parse the headers 1464 foreach(h; pps.thisOnesHeaders) { 1465 auto p = h.indexOf(":"); 1466 assert(p != -1, "no colon in header, got " ~ to!string(pps.thisOnesHeaders)); 1467 string hn = h[0..p]; 1468 string hv = h[p+2..$]; 1469 1470 switch(hn.toLower) { 1471 default: assert(0); 1472 case "content-disposition": 1473 auto info = hv.split("; "); 1474 foreach(i; info[1..$]) { // skipping the form-data 1475 auto o = i.split("="); // FIXME 1476 string pn = o[0]; 1477 string pv = o[1][1..$-1]; 1478 1479 if(pn == "name") { 1480 pps.piece.name = pv; 1481 } else if (pn == "filename") { 1482 pps.piece.filename = pv; 1483 pps.isFile = true; 1484 } 1485 } 1486 break; 1487 case "content-type": 1488 pps.piece.contentType = hv; 1489 break; 1490 } 1491 } 1492 1493 pps.whatDoWeWant++; // move to the next step - the data 1494 break; 1495 case 2: 1496 // when we get here, pps.buffer should contain our first chunk of data 1497 1498 if(pps.buffer.length + chunk.length > 8 * 1024 * 1024) // we might buffer quite a bit but not much 1499 throw new Exception("wtf is up with the huge mime part buffer"); 1500 1501 acceptChunk(); 1502 1503 // so the trick is, we want to process all the data up to the boundary, 1504 // but what if the chunk's end cuts the boundary off? If we're unsure, we 1505 // want to wait for the next chunk. We start by looking for the whole boundary 1506 // in the buffer somewhere. 1507 1508 auto boundaryLocation = locationOf(pps.buffer, pps.localBoundary); 1509 // assert(boundaryLocation != -1, "should have seen "~to!string(cast(ubyte[]) pps.localBoundary)~" in " ~ to!string(pps.buffer)); 1510 if(boundaryLocation != -1) { 1511 // this is easy - we can see it in it's entirety! 1512 1513 pps.piece.content ~= consume(boundaryLocation); 1514 1515 assert(pps.buffer[0] == '\r'); 1516 consume(1); 1517 assert(pps.buffer[0] == '\n'); 1518 consume(1); 1519 assert(pps.buffer[0] == '-'); 1520 consume(1); 1521 assert(pps.buffer[0] == '-'); 1522 consume(1); 1523 // the boundary here is always preceded by \r\n--, which is why we used localBoundary instead of boundary to locate it. Cut that off. 1524 pps.weHaveAPart = true; 1525 pps.whatDoWeWant = 1; // back to getting headers for the next part 1526 1527 commitPart(); // we're done here 1528 } else { 1529 // we can't see the whole thing, but what if there's a partial boundary? 1530 1531 enforce(pps.localBoundary.length < 128); // the boundary ought to be less than a line... 1532 assert(pps.localBoundary.length > 1); // should already be sane but just in case 1533 bool potentialBoundaryFound = false; 1534 1535 boundaryCheck: for(int a = 1; a < pps.localBoundary.length; a++) { 1536 // we grow the boundary a bit each time. If we think it looks the 1537 // same, better pull another chunk to be sure it's not the end. 1538 // Starting small because exiting the loop early is desirable, since 1539 // we're not keeping any ambiguity and 1 / 256 chance of exiting is 1540 // the best we can do. 1541 if(a > pps.buffer.length) 1542 break; // FIXME: is this right? 1543 assert(a <= pps.buffer.length); 1544 assert(a > 0); 1545 if(std.algorithm.endsWith(pps.buffer, pps.localBoundary[0 .. a])) { 1546 // ok, there *might* be a boundary here, so let's 1547 // not treat the end as data yet. The rest is good to 1548 // use though, since if there was a boundary there, we'd 1549 // have handled it up above after locationOf. 1550 1551 pps.piece.content ~= pps.buffer[0 .. $ - a]; 1552 consume(pps.buffer.length - a); 1553 pieceHasNewContent(); 1554 potentialBoundaryFound = true; 1555 break boundaryCheck; 1556 } 1557 } 1558 1559 if(!potentialBoundaryFound) { 1560 // we can consume the whole thing 1561 pps.piece.content ~= pps.buffer; 1562 pieceHasNewContent(); 1563 consume(pps.buffer.length); 1564 } else { 1565 // we found a possible boundary, but there was 1566 // insufficient data to be sure. 1567 assert(pps.buffer == cast(const(ubyte[])) pps.localBoundary[0 .. pps.buffer.length]); 1568 1569 return; // wait for the next chunk. 1570 } 1571 } 1572 } 1573 } while(pps.buffer.length); 1574 1575 // btw all boundaries except the first should have a \r\n before them 1576 } else { 1577 // application/x-www-form-urlencoded and application/json 1578 1579 // not using maxContentLength because that might be cranked up to allow 1580 // large file uploads. We can handle them, but a huge post[] isn't any good. 1581 if(pps.buffer.length + chunk.length > 8 * 1024 * 1024) // surely this is plenty big enough 1582 throw new Exception("wtf is up with such a gigantic form submission????"); 1583 1584 pps.buffer ~= chunk; 1585 1586 // simple handling, but it works... until someone bombs us with gigabytes of crap at least... 1587 if(pps.buffer.length == pps.expectedLength) { 1588 if(pps.needsSavedBody) 1589 pps.postBody = cast(string) pps.buffer; 1590 else 1591 pps._post = decodeVariables(cast(string) pps.buffer, "&", &allPostNamesInOrder, &allPostValuesInOrder); 1592 version(preserveData) 1593 originalPostData = pps.buffer; 1594 } else { 1595 // just for debugging 1596 } 1597 } 1598 } 1599 1600 protected void cleanUpPostDataState() { 1601 pps = PostParserState.init; 1602 } 1603 1604 /// you can override this function to somehow react 1605 /// to an upload in progress. 1606 /// 1607 /// Take note that parts of the CGI object is not yet 1608 /// initialized! Stuff from HTTP headers, including get[], is usable. 1609 /// But, none of post[] is usable, and you cannot write here. That's 1610 /// why this method is const - mutating the object won't do much anyway. 1611 /// 1612 /// My idea here was so you can output a progress bar or 1613 /// something to a cooperative client (see arsd.rtud for a potential helper) 1614 /// 1615 /// The default is to do nothing. Subclass cgi and use the 1616 /// CustomCgiMain mixin to do something here. 1617 void onRequestBodyDataReceived(size_t receivedSoFar, size_t totalExpected) const { 1618 // This space intentionally left blank. 1619 } 1620 1621 /// Initializes the cgi from completely raw HTTP data. The ir must have a Socket source. 1622 /// *closeConnection will be set to true if you should close the connection after handling this request 1623 this(BufferedInputRange ir, bool* closeConnection) { 1624 isCalledWithCommandLineArguments = false; 1625 import al = std.algorithm; 1626 1627 immutable(ubyte)[] data; 1628 1629 void rdo(const(ubyte)[] d) { 1630 //import std.stdio; writeln(d); 1631 sendAll(ir.source, d); 1632 } 1633 1634 auto ira = ir.source.remoteAddress(); 1635 auto irLocalAddress = ir.source.localAddress(); 1636 1637 ushort port = 80; 1638 if(auto ia = cast(InternetAddress) irLocalAddress) { 1639 port = ia.port; 1640 } else if(auto ia = cast(Internet6Address) irLocalAddress) { 1641 port = ia.port; 1642 } 1643 1644 // that check for UnixAddress is to work around a Phobos bug 1645 // see: https://github.com/dlang/phobos/pull/7383 1646 // but this might be more useful anyway tbh for this case 1647 version(Posix) 1648 this(ir, ira is null ? null : cast(UnixAddress) ira ? "unix:" : ira.toString(), port, 0, false, &rdo, null, closeConnection); 1649 else 1650 this(ir, ira is null ? null : ira.toString(), port, 0, false, &rdo, null, closeConnection); 1651 } 1652 1653 /** 1654 Initializes it from raw HTTP request data. GenericMain uses this when you compile with -version=embedded_httpd. 1655 1656 NOTE: If you are behind a reverse proxy, the values here might not be what you expect.... it will use X-Forwarded-For for remote IP and X-Forwarded-Host for host 1657 1658 Params: 1659 inputData = the incoming data, including headers and other raw http data. 1660 When the constructor exits, it will leave this range exactly at the start of 1661 the next request on the connection (if there is one). 1662 1663 address = the IP address of the remote user 1664 _port = the port number of the connection 1665 pathInfoStarts = the offset into the path component of the http header where the SCRIPT_NAME ends and the PATH_INFO begins. 1666 _https = if this connection is encrypted (note that the input data must not actually be encrypted) 1667 _rawDataOutput = delegate to accept response data. It should write to the socket or whatever; Cgi does all the needed processing to speak http. 1668 _flush = if _rawDataOutput buffers, this delegate should flush the buffer down the wire 1669 closeConnection = if the request asks to close the connection, *closeConnection == true. 1670 */ 1671 this( 1672 BufferedInputRange inputData, 1673 // string[] headers, immutable(ubyte)[] data, 1674 string address, ushort _port, 1675 int pathInfoStarts = 0, // use this if you know the script name, like if this is in a folder in a bigger web environment 1676 bool _https = false, 1677 void delegate(const(ubyte)[]) _rawDataOutput = null, 1678 void delegate() _flush = null, 1679 // this pointer tells if the connection is supposed to be closed after we handle this 1680 bool* closeConnection = null) 1681 { 1682 // these are all set locally so the loop works 1683 // without triggering errors in dmd 2.064 1684 // we go ahead and set them at the end of it to the this version 1685 int port; 1686 string referrer; 1687 string remoteAddress; 1688 string userAgent; 1689 string authorization; 1690 string origin; 1691 string accept; 1692 string lastEventId; 1693 bool https; 1694 string host; 1695 RequestMethod requestMethod; 1696 string requestUri; 1697 string pathInfo; 1698 string queryString; 1699 string scriptName; 1700 string[string] get; 1701 string[][string] getArray; 1702 bool keepAliveRequested; 1703 bool acceptsGzip; 1704 string cookie; 1705 1706 1707 1708 environmentVariables = cast(const) environment.toAA; 1709 1710 idlol = inputData; 1711 1712 isCalledWithCommandLineArguments = false; 1713 1714 https = _https; 1715 port = _port; 1716 1717 rawDataOutput = _rawDataOutput; 1718 flushDelegate = _flush; 1719 nph = true; 1720 1721 remoteAddress = address; 1722 1723 // streaming parser 1724 import al = std.algorithm; 1725 1726 // FIXME: tis cast is technically wrong, but Phobos deprecated al.indexOf... for some reason. 1727 auto idx = indexOf(cast(string) inputData.front(), "\r\n\r\n"); 1728 while(idx == -1) { 1729 inputData.popFront(0); 1730 idx = indexOf(cast(string) inputData.front(), "\r\n\r\n"); 1731 } 1732 1733 assert(idx != -1); 1734 1735 1736 string contentType = ""; 1737 string[string] requestHeadersHere; 1738 1739 size_t contentLength; 1740 1741 bool isChunked; 1742 1743 { 1744 import core.runtime; 1745 scriptFileName = Runtime.args[0]; 1746 } 1747 1748 1749 int headerNumber = 0; 1750 foreach(line; al.splitter(inputData.front()[0 .. idx], "\r\n")) 1751 if(line.length) { 1752 headerNumber++; 1753 auto header = cast(string) line.idup; 1754 if(headerNumber == 1) { 1755 // request line 1756 auto parts = al.splitter(header, " "); 1757 requestMethod = to!RequestMethod(parts.front); 1758 parts.popFront(); 1759 requestUri = parts.front; 1760 1761 // FIXME: the requestUri could be an absolute path!!! should I rename it or something? 1762 scriptName = requestUri[0 .. pathInfoStarts]; 1763 1764 auto question = requestUri.indexOf("?"); 1765 if(question == -1) { 1766 queryString = ""; 1767 // FIXME: double check, this might be wrong since it could be url encoded 1768 pathInfo = requestUri[pathInfoStarts..$]; 1769 } else { 1770 queryString = requestUri[question+1..$]; 1771 pathInfo = requestUri[pathInfoStarts..question]; 1772 } 1773 1774 auto ugh = decodeVariables(queryString, "&", &allGetNamesInOrder, &allGetValuesInOrder); 1775 getArray = cast(string[][string]) assumeUnique(ugh); 1776 1777 if(header.indexOf("HTTP/1.0") != -1) { 1778 http10 = true; 1779 autoBuffer = true; 1780 if(closeConnection) { 1781 // on http 1.0, close is assumed (unlike http/1.1 where we assume keep alive) 1782 *closeConnection = true; 1783 } 1784 } 1785 } else { 1786 // other header 1787 auto colon = header.indexOf(":"); 1788 if(colon == -1) 1789 throw new Exception("HTTP headers should have a colon!"); 1790 string name = header[0..colon].toLower; 1791 string value = header[colon+2..$]; // skip the colon and the space 1792 1793 requestHeadersHere[name] = value; 1794 1795 if (name == "accept") { 1796 accept = value; 1797 } 1798 else if (name == "origin") { 1799 origin = value; 1800 } 1801 else if (name == "connection") { 1802 if(value == "close" && closeConnection) 1803 *closeConnection = true; 1804 if(value.asLowerCase().canFind("keep-alive")) { 1805 keepAliveRequested = true; 1806 1807 // on http 1.0, the connection is closed by default, 1808 // but not if they request keep-alive. then we don't close 1809 // anymore - undoing the set above 1810 if(http10 && closeConnection) { 1811 *closeConnection = false; 1812 } 1813 } 1814 } 1815 else if (name == "transfer-encoding") { 1816 if(value == "chunked") 1817 isChunked = true; 1818 } 1819 else if (name == "last-event-id") { 1820 lastEventId = value; 1821 } 1822 else if (name == "authorization") { 1823 authorization = value; 1824 } 1825 else if (name == "content-type") { 1826 contentType = value; 1827 } 1828 else if (name == "content-length") { 1829 contentLength = to!size_t(value); 1830 } 1831 else if (name == "x-forwarded-for") { 1832 remoteAddress = value; 1833 } 1834 else if (name == "x-forwarded-host" || name == "host") { 1835 if(name != "host" || host is null) 1836 host = value; 1837 } 1838 // FIXME: https://tools.ietf.org/html/rfc7239 1839 else if (name == "accept-encoding") { 1840 if(value.indexOf("gzip") != -1) 1841 acceptsGzip = true; 1842 } 1843 else if (name == "user-agent") { 1844 userAgent = value; 1845 } 1846 else if (name == "referer") { 1847 referrer = value; 1848 } 1849 else if (name == "cookie") { 1850 cookie ~= value; 1851 } else if(name == "expect") { 1852 if(value == "100-continue") { 1853 // FIXME we should probably give user code a chance 1854 // to process and reject but that needs to be virtual, 1855 // perhaps part of the CGI redesign. 1856 1857 // FIXME: if size is > max content length it should 1858 // also fail at this point. 1859 _rawDataOutput(cast(ubyte[]) "HTTP/1.1 100 Continue\r\n\r\n"); 1860 } 1861 } 1862 // else 1863 // ignore it 1864 1865 } 1866 } 1867 1868 inputData.consume(idx + 4); 1869 // done 1870 1871 requestHeaders = assumeUnique(requestHeadersHere); 1872 1873 ByChunkRange dataByChunk; 1874 1875 // reading Content-Length type data 1876 // We need to read up the data we have, and write it out as a chunk. 1877 if(!isChunked) { 1878 dataByChunk = byChunk(inputData, contentLength); 1879 } else { 1880 // chunked requests happen, but not every day. Since we need to know 1881 // the content length (for now, maybe that should change), we'll buffer 1882 // the whole thing here instead of parse streaming. (I think this is what Apache does anyway in cgi modes) 1883 auto data = dechunk(inputData); 1884 1885 // set the range here 1886 dataByChunk = byChunk(data); 1887 contentLength = data.length; 1888 } 1889 1890 assert(dataByChunk !is null); 1891 1892 if(contentLength) { 1893 prepareForIncomingDataChunks(contentType, contentLength); 1894 foreach(dataChunk; dataByChunk) { 1895 handleIncomingDataChunk(dataChunk); 1896 } 1897 postArray = assumeUnique(pps._post); 1898 filesArray = assumeUnique(pps._files); 1899 files = keepLastOf(filesArray); 1900 post = keepLastOf(postArray); 1901 postBody = pps.postBody; 1902 cleanUpPostDataState(); 1903 } 1904 1905 this.port = port; 1906 this.referrer = referrer; 1907 this.remoteAddress = remoteAddress; 1908 this.userAgent = userAgent; 1909 this.authorization = authorization; 1910 this.origin = origin; 1911 this.accept = accept; 1912 this.lastEventId = lastEventId; 1913 this.https = https; 1914 this.host = host; 1915 this.requestMethod = requestMethod; 1916 this.requestUri = requestUri; 1917 this.pathInfo = pathInfo; 1918 this.queryString = queryString; 1919 1920 this.scriptName = scriptName; 1921 this.get = keepLastOf(getArray); 1922 this.getArray = cast(immutable) getArray; 1923 this.keepAliveRequested = keepAliveRequested; 1924 this.acceptsGzip = acceptsGzip; 1925 this.cookie = cookie; 1926 1927 cookiesArray = getCookieArray(); 1928 cookies = keepLastOf(cookiesArray); 1929 1930 } 1931 BufferedInputRange idlol; 1932 1933 private immutable(string[string]) keepLastOf(in string[][string] arr) { 1934 string[string] ca; 1935 foreach(k, v; arr) 1936 ca[k] = v[$-1]; 1937 1938 return assumeUnique(ca); 1939 } 1940 1941 // FIXME duplication 1942 private immutable(UploadedFile[string]) keepLastOf(in UploadedFile[][string] arr) { 1943 UploadedFile[string] ca; 1944 foreach(k, v; arr) 1945 ca[k] = v[$-1]; 1946 1947 return assumeUnique(ca); 1948 } 1949 1950 1951 private immutable(string[][string]) getCookieArray() { 1952 auto forTheLoveOfGod = decodeVariables(cookie, "; "); 1953 return assumeUnique(forTheLoveOfGod); 1954 } 1955 1956 /// Very simple method to require a basic auth username and password. 1957 /// If the http request doesn't include the required credentials, it throws a 1958 /// HTTP 401 error, and an exception. 1959 /// 1960 /// Note: basic auth does not provide great security, especially over unencrypted HTTP; 1961 /// the user's credentials are sent in plain text on every request. 1962 /// 1963 /// If you are using Apache, the HTTP_AUTHORIZATION variable may not be sent to the 1964 /// application. Either use Apache's built in methods for basic authentication, or add 1965 /// something along these lines to your server configuration: 1966 /// 1967 /// RewriteEngine On 1968 /// RewriteCond %{HTTP:Authorization} ^(.*) 1969 /// RewriteRule ^(.*) - [E=HTTP_AUTHORIZATION:%1] 1970 /// 1971 /// To ensure the necessary data is available to cgi.d. 1972 void requireBasicAuth(string user, string pass, string message = null) { 1973 if(authorization != "Basic " ~ Base64.encode(cast(immutable(ubyte)[]) (user ~ ":" ~ pass))) { 1974 setResponseStatus("401 Authorization Required"); 1975 header ("WWW-Authenticate: Basic realm=\""~message~"\""); 1976 close(); 1977 throw new Exception("Not authorized; got " ~ authorization); 1978 } 1979 } 1980 1981 /// Very simple caching controls - setCache(false) means it will never be cached. Good for rapidly updated or sensitive sites. 1982 /// setCache(true) means it will always be cached for as long as possible. Best for static content. 1983 /// Use setResponseExpires and updateResponseExpires for more control 1984 void setCache(bool allowCaching) { 1985 noCache = !allowCaching; 1986 } 1987 1988 /// Set to true and use cgi.write(data, true); to send a gzipped response to browsers 1989 /// who can accept it 1990 bool gzipResponse; 1991 1992 immutable bool acceptsGzip; 1993 immutable bool keepAliveRequested; 1994 1995 /// Set to true if and only if this was initialized with command line arguments 1996 immutable bool isCalledWithCommandLineArguments; 1997 1998 /// This gets a full url for the current request, including port, protocol, host, path, and query 1999 string getCurrentCompleteUri() const { 2000 ushort defaultPort = https ? 443 : 80; 2001 2002 string uri = "http"; 2003 if(https) 2004 uri ~= "s"; 2005 uri ~= "://"; 2006 uri ~= host; 2007 if(!(!port || port == defaultPort)) { 2008 uri ~= ":"; 2009 uri ~= to!string(port); 2010 } 2011 uri ~= requestUri; 2012 return uri; 2013 } 2014 2015 /// You can override this if your site base url isn't the same as the script name 2016 string logicalScriptName() const { 2017 return scriptName; 2018 } 2019 2020 /++ 2021 Sets the HTTP status of the response. For example, "404 File Not Found" or "500 Internal Server Error". 2022 It assumes "200 OK", and automatically changes to "302 Found" if you call setResponseLocation(). 2023 Note setResponseStatus() must be called *before* you write() any data to the output. 2024 2025 History: 2026 The `int` overload was added on January 11, 2021. 2027 +/ 2028 void setResponseStatus(string status) { 2029 assert(!outputtedResponseData); 2030 responseStatus = status; 2031 } 2032 /// ditto 2033 void setResponseStatus(int statusCode) { 2034 setResponseStatus(getHttpCodeText(statusCode)); 2035 } 2036 private string responseStatus = null; 2037 2038 /// Returns true if it is still possible to output headers 2039 bool canOutputHeaders() { 2040 return !isClosed && !outputtedResponseData; 2041 } 2042 2043 /// Sets the location header, which the browser will redirect the user to automatically. 2044 /// Note setResponseLocation() must be called *before* you write() any data to the output. 2045 /// The optional important argument is used if it's a default suggestion rather than something to insist upon. 2046 void setResponseLocation(string uri, bool important = true, string status = null) { 2047 if(!important && isCurrentResponseLocationImportant) 2048 return; // important redirects always override unimportant ones 2049 2050 if(uri is null) { 2051 responseStatus = "200 OK"; 2052 responseLocation = null; 2053 isCurrentResponseLocationImportant = important; 2054 return; // this just cancels the redirect 2055 } 2056 2057 assert(!outputtedResponseData); 2058 if(status is null) 2059 responseStatus = "302 Found"; 2060 else 2061 responseStatus = status; 2062 2063 responseLocation = uri.strip; 2064 isCurrentResponseLocationImportant = important; 2065 } 2066 protected string responseLocation = null; 2067 private bool isCurrentResponseLocationImportant = false; 2068 2069 /// Sets the Expires: http header. See also: updateResponseExpires, setPublicCaching 2070 /// The parameter is in unix_timestamp * 1000. Try setResponseExpires(getUTCtime() + SOME AMOUNT) for normal use. 2071 /// Note: the when parameter is different than setCookie's expire parameter. 2072 void setResponseExpires(long when, bool isPublic = false) { 2073 responseExpires = when; 2074 setCache(true); // need to enable caching so the date has meaning 2075 2076 responseIsPublic = isPublic; 2077 responseExpiresRelative = false; 2078 } 2079 2080 /// Sets a cache-control max-age header for whenFromNow, in seconds. 2081 void setResponseExpiresRelative(int whenFromNow, bool isPublic = false) { 2082 responseExpires = whenFromNow; 2083 setCache(true); // need to enable caching so the date has meaning 2084 2085 responseIsPublic = isPublic; 2086 responseExpiresRelative = true; 2087 } 2088 private long responseExpires = long.min; 2089 private bool responseIsPublic = false; 2090 private bool responseExpiresRelative = false; 2091 2092 /// This is like setResponseExpires, but it can be called multiple times. The setting most in the past is the one kept. 2093 /// If you have multiple functions, they all might call updateResponseExpires about their own return value. The program 2094 /// output as a whole is as cacheable as the least cachable part in the chain. 2095 2096 /// setCache(false) always overrides this - it is, by definition, the strictest anti-cache statement available. If your site outputs sensitive user data, you should probably call setCache(false) when you do, to ensure no other functions will cache the content, as it may be a privacy risk. 2097 /// Conversely, setting here overrides setCache(true), since any expiration date is in the past of infinity. 2098 void updateResponseExpires(long when, bool isPublic) { 2099 if(responseExpires == long.min) 2100 setResponseExpires(when, isPublic); 2101 else if(when < responseExpires) 2102 setResponseExpires(when, responseIsPublic && isPublic); // if any part of it is private, it all is 2103 } 2104 2105 /* 2106 /// Set to true if you want the result to be cached publically - that is, is the content shared? 2107 /// Should generally be false if the user is logged in. It assumes private cache only. 2108 /// setCache(true) also turns on public caching, and setCache(false) sets to private. 2109 void setPublicCaching(bool allowPublicCaches) { 2110 publicCaching = allowPublicCaches; 2111 } 2112 private bool publicCaching = false; 2113 */ 2114 2115 /++ 2116 History: 2117 Added January 11, 2021 2118 +/ 2119 enum SameSitePolicy { 2120 Lax, 2121 Strict, 2122 None 2123 } 2124 2125 /++ 2126 Sets an HTTP cookie, automatically encoding the data to the correct string. 2127 expiresIn is how many milliseconds in the future the cookie will expire. 2128 TIP: to make a cookie accessible from subdomains, set the domain to .yourdomain.com. 2129 Note setCookie() must be called *before* you write() any data to the output. 2130 2131 History: 2132 Parameter `sameSitePolicy` was added on January 11, 2021. 2133 +/ 2134 void setCookie(string name, string data, long expiresIn = 0, string path = null, string domain = null, bool httpOnly = false, bool secure = false, SameSitePolicy sameSitePolicy = SameSitePolicy.Lax) { 2135 assert(!outputtedResponseData); 2136 string cookie = std.uri.encodeComponent(name) ~ "="; 2137 cookie ~= std.uri.encodeComponent(data); 2138 if(path !is null) 2139 cookie ~= "; path=" ~ path; 2140 // FIXME: should I just be using max-age here? (also in cache below) 2141 if(expiresIn != 0) 2142 cookie ~= "; expires=" ~ printDate(cast(DateTime) Clock.currTime(UTC()) + dur!"msecs"(expiresIn)); 2143 if(domain !is null) 2144 cookie ~= "; domain=" ~ domain; 2145 if(secure == true) 2146 cookie ~= "; Secure"; 2147 if(httpOnly == true ) 2148 cookie ~= "; HttpOnly"; 2149 final switch(sameSitePolicy) { 2150 case SameSitePolicy.Lax: 2151 cookie ~= "; SameSite=Lax"; 2152 break; 2153 case SameSitePolicy.Strict: 2154 cookie ~= "; SameSite=Strict"; 2155 break; 2156 case SameSitePolicy.None: 2157 cookie ~= "; SameSite=None"; 2158 assert(secure); // cookie spec requires this now, see: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite 2159 break; 2160 } 2161 2162 if(auto idx = name in cookieIndexes) { 2163 responseCookies[*idx] = cookie; 2164 } else { 2165 cookieIndexes[name] = responseCookies.length; 2166 responseCookies ~= cookie; 2167 } 2168 } 2169 private string[] responseCookies; 2170 private size_t[string] cookieIndexes; 2171 2172 /// Clears a previously set cookie with the given name, path, and domain. 2173 void clearCookie(string name, string path = null, string domain = null) { 2174 assert(!outputtedResponseData); 2175 setCookie(name, "", 1, path, domain); 2176 } 2177 2178 /// Sets the content type of the response, for example "text/html" (the default) for HTML, or "image/png" for a PNG image 2179 void setResponseContentType(string ct) { 2180 assert(!outputtedResponseData); 2181 responseContentType = ct; 2182 } 2183 private string responseContentType = null; 2184 2185 /// Adds a custom header. It should be the name: value, but without any line terminator. 2186 /// For example: header("X-My-Header: Some value"); 2187 /// Note you should use the specialized functions in this object if possible to avoid 2188 /// duplicates in the output. 2189 void header(string h) { 2190 customHeaders ~= h; 2191 } 2192 2193 private string[] customHeaders; 2194 private bool websocketMode; 2195 2196 void flushHeaders(const(void)[] t, bool isAll = false) { 2197 StackBuffer buffer = StackBuffer(0); 2198 2199 prepHeaders(t, isAll, &buffer); 2200 2201 if(rawDataOutput !is null) 2202 rawDataOutput(cast(const(ubyte)[]) buffer.get()); 2203 else { 2204 stdout.rawWrite(buffer.get()); 2205 } 2206 } 2207 2208 private void prepHeaders(const(void)[] t, bool isAll, StackBuffer* buffer) { 2209 string terminator = "\n"; 2210 if(rawDataOutput !is null) 2211 terminator = "\r\n"; 2212 2213 if(responseStatus !is null) { 2214 if(nph) { 2215 if(http10) 2216 buffer.add("HTTP/1.0 ", responseStatus, terminator); 2217 else 2218 buffer.add("HTTP/1.1 ", responseStatus, terminator); 2219 } else 2220 buffer.add("Status: ", responseStatus, terminator); 2221 } else if (nph) { 2222 if(http10) 2223 buffer.add("HTTP/1.0 200 OK", terminator); 2224 else 2225 buffer.add("HTTP/1.1 200 OK", terminator); 2226 } 2227 2228 if(websocketMode) 2229 goto websocket; 2230 2231 if(nph) { // we're responsible for setting the date too according to http 1.1 2232 char[29] db = void; 2233 printDateToBuffer(cast(DateTime) Clock.currTime(UTC()), db[]); 2234 buffer.add("Date: ", db[], terminator); 2235 } 2236 2237 // FIXME: what if the user wants to set his own content-length? 2238 // The custom header function can do it, so maybe that's best. 2239 // Or we could reuse the isAll param. 2240 if(responseLocation !is null) { 2241 buffer.add("Location: ", responseLocation, terminator); 2242 } 2243 if(!noCache && responseExpires != long.min) { // an explicit expiration date is set 2244 if(responseExpiresRelative) { 2245 buffer.add("Cache-Control: ", responseIsPublic ? "public" : "private", ", max-age="); 2246 buffer.add(responseExpires); 2247 buffer.add(", no-cache=\"set-cookie, set-cookie2\"", terminator); 2248 } else { 2249 auto expires = SysTime(unixTimeToStdTime(cast(int)(responseExpires / 1000)), UTC()); 2250 char[29] db = void; 2251 printDateToBuffer(cast(DateTime) expires, db[]); 2252 buffer.add("Expires: ", db[], terminator); 2253 // FIXME: assuming everything is private unless you use nocache - generally right for dynamic pages, but not necessarily 2254 buffer.add("Cache-Control: ", (responseIsPublic ? "public" : "private"), ", no-cache=\"set-cookie, set-cookie2\""); 2255 buffer.add(terminator); 2256 } 2257 } 2258 if(responseCookies !is null && responseCookies.length > 0) { 2259 foreach(c; responseCookies) 2260 buffer.add("Set-Cookie: ", c, terminator); 2261 } 2262 if(noCache) { // we specifically do not want caching (this is actually the default) 2263 buffer.add("Cache-Control: private, no-cache=\"set-cookie\"", terminator); 2264 buffer.add("Expires: 0", terminator); 2265 buffer.add("Pragma: no-cache", terminator); 2266 } else { 2267 if(responseExpires == long.min) { // caching was enabled, but without a date set - that means assume cache forever 2268 buffer.add("Cache-Control: public", terminator); 2269 buffer.add("Expires: Tue, 31 Dec 2030 14:00:00 GMT", terminator); // FIXME: should not be more than one year in the future 2270 } 2271 } 2272 if(responseContentType !is null) { 2273 buffer.add("Content-Type: ", responseContentType, terminator); 2274 } else 2275 buffer.add("Content-Type: text/html; charset=utf-8", terminator); 2276 2277 if(gzipResponse && acceptsGzip && isAll) { // FIXME: isAll really shouldn't be necessary 2278 buffer.add("Content-Encoding: gzip", terminator); 2279 } 2280 2281 2282 if(!isAll) { 2283 if(nph && !http10) { 2284 buffer.add("Transfer-Encoding: chunked", terminator); 2285 responseChunked = true; 2286 } 2287 } else { 2288 buffer.add("Content-Length: "); 2289 buffer.add(t.length); 2290 buffer.add(terminator); 2291 if(nph && keepAliveRequested) { 2292 buffer.add("Connection: Keep-Alive", terminator); 2293 } 2294 } 2295 2296 websocket: 2297 2298 foreach(hd; customHeaders) 2299 buffer.add(hd, terminator); 2300 2301 // FIXME: what about duplicated headers? 2302 2303 // end of header indicator 2304 buffer.add(terminator); 2305 2306 outputtedResponseData = true; 2307 } 2308 2309 /// Writes the data to the output, flushing headers if they have not yet been sent. 2310 void write(const(void)[] t, bool isAll = false, bool maybeAutoClose = true) { 2311 assert(!closed, "Output has already been closed"); 2312 2313 StackBuffer buffer = StackBuffer(0); 2314 2315 if(gzipResponse && acceptsGzip && isAll) { // FIXME: isAll really shouldn't be necessary 2316 // actually gzip the data here 2317 2318 auto c = new Compress(HeaderFormat.gzip); // want gzip 2319 2320 auto data = c.compress(t); 2321 data ~= c.flush(); 2322 2323 // std.file.write("/tmp/last-item", data); 2324 2325 t = data; 2326 } 2327 2328 if(!outputtedResponseData && (!autoBuffer || isAll)) { 2329 prepHeaders(t, isAll, &buffer); 2330 } 2331 2332 if(requestMethod != RequestMethod.HEAD && t.length > 0) { 2333 if (autoBuffer && !isAll) { 2334 outputBuffer ~= cast(ubyte[]) t; 2335 } 2336 if(!autoBuffer || isAll) { 2337 if(rawDataOutput !is null) 2338 if(nph && responseChunked) { 2339 //rawDataOutput(makeChunk(cast(const(ubyte)[]) t)); 2340 // we're making the chunk here instead of in a function 2341 // to avoid unneeded gc pressure 2342 buffer.add(toHex(t.length)); 2343 buffer.add("\r\n"); 2344 buffer.add(cast(char[]) t, "\r\n"); 2345 } else { 2346 buffer.add(cast(char[]) t); 2347 } 2348 else 2349 buffer.add(cast(char[]) t); 2350 } 2351 } 2352 2353 if(rawDataOutput !is null) 2354 rawDataOutput(cast(const(ubyte)[]) buffer.get()); 2355 else 2356 stdout.rawWrite(buffer.get()); 2357 2358 if(maybeAutoClose && isAll) 2359 close(); // if you say it is all, that means we're definitely done 2360 // maybeAutoClose can be false though to avoid this (important if you call from inside close()! 2361 } 2362 2363 /++ 2364 Convenience method to set content type to json and write the string as the complete response. 2365 2366 History: 2367 Added January 16, 2020 2368 +/ 2369 void writeJson(string json) { 2370 this.setResponseContentType("application/json"); 2371 this.write(json, true); 2372 } 2373 2374 /// Flushes the pending buffer, leaving the connection open so you can send more. 2375 void flush() { 2376 if(rawDataOutput is null) 2377 stdout.flush(); 2378 else if(flushDelegate !is null) 2379 flushDelegate(); 2380 } 2381 2382 version(autoBuffer) 2383 bool autoBuffer = true; 2384 else 2385 bool autoBuffer = false; 2386 ubyte[] outputBuffer; 2387 2388 /// Flushes the buffers to the network, signifying that you are done. 2389 /// You should always call this explicitly when you are done outputting data. 2390 void close() { 2391 if(closed) 2392 return; // don't double close 2393 2394 if(!outputtedResponseData) 2395 write("", false, false); 2396 2397 // writing auto buffered data 2398 if(requestMethod != RequestMethod.HEAD && autoBuffer) { 2399 if(!nph) 2400 stdout.rawWrite(outputBuffer); 2401 else 2402 write(outputBuffer, true, false); // tell it this is everything 2403 } 2404 2405 // closing the last chunk... 2406 if(nph && rawDataOutput !is null && responseChunked) 2407 rawDataOutput(cast(const(ubyte)[]) "0\r\n\r\n"); 2408 2409 if(flushDelegate) 2410 flushDelegate(); 2411 2412 closed = true; 2413 } 2414 2415 // Closes without doing anything, shouldn't be used often 2416 void rawClose() { 2417 closed = true; 2418 } 2419 2420 /++ 2421 Gets a request variable as a specific type, or the default value of it isn't there 2422 or isn't convertible to the request type. 2423 2424 Checks both GET and POST variables, preferring the POST variable, if available. 2425 2426 A nice trick is using the default value to choose the type: 2427 2428 --- 2429 /* 2430 The return value will match the type of the default. 2431 Here, I gave 10 as a default, so the return value will 2432 be an int. 2433 2434 If the user-supplied value cannot be converted to the 2435 requested type, you will get the default value back. 2436 */ 2437 int a = cgi.request("number", 10); 2438 2439 if(cgi.get["number"] == "11") 2440 assert(a == 11); // conversion succeeds 2441 2442 if("number" !in cgi.get) 2443 assert(a == 10); // no value means you can't convert - give the default 2444 2445 if(cgi.get["number"] == "twelve") 2446 assert(a == 10); // conversion from string to int would fail, so we get the default 2447 --- 2448 2449 You can use an enum as an easy whitelist, too: 2450 2451 --- 2452 enum Operations { 2453 add, remove, query 2454 } 2455 2456 auto op = cgi.request("op", Operations.query); 2457 2458 if(cgi.get["op"] == "add") 2459 assert(op == Operations.add); 2460 if(cgi.get["op"] == "remove") 2461 assert(op == Operations.remove); 2462 if(cgi.get["op"] == "query") 2463 assert(op == Operations.query); 2464 2465 if(cgi.get["op"] == "random string") 2466 assert(op == Operations.query); // the value can't be converted to the enum, so we get the default 2467 --- 2468 +/ 2469 T request(T = string)(in string name, in T def = T.init) const nothrow { 2470 try { 2471 return 2472 (name in post) ? to!T(post[name]) : 2473 (name in get) ? to!T(get[name]) : 2474 def; 2475 } catch(Exception e) { return def; } 2476 } 2477 2478 /// Is the output already closed? 2479 bool isClosed() const { 2480 return closed; 2481 } 2482 2483 /++ 2484 Gets a session object associated with the `cgi` request. You can use different type throughout your application. 2485 +/ 2486 Session!Data getSessionObject(Data)() { 2487 if(testInProcess !is null) { 2488 // test mode 2489 auto obj = testInProcess.getSessionOverride(typeid(typeof(return))); 2490 if(obj !is null) 2491 return cast(typeof(return)) obj; 2492 else { 2493 auto o = new MockSession!Data(); 2494 testInProcess.setSessionOverride(typeid(typeof(return)), o); 2495 return o; 2496 } 2497 } else { 2498 // normal operation 2499 return new BasicDataServerSession!Data(this); 2500 } 2501 } 2502 2503 // if it is in test mode; triggers mock sessions. Used by CgiTester 2504 version(with_breaking_cgi_features) 2505 private CgiTester testInProcess; 2506 2507 /* Hooks for redirecting input and output */ 2508 private void delegate(const(ubyte)[]) rawDataOutput = null; 2509 private void delegate() flushDelegate = null; 2510 2511 /* This info is used when handling a more raw HTTP protocol */ 2512 private bool nph; 2513 private bool http10; 2514 private bool closed; 2515 private bool responseChunked = false; 2516 2517 version(preserveData) // note: this can eat lots of memory; don't use unless you're sure you need it. 2518 immutable(ubyte)[] originalPostData; 2519 2520 public immutable string postBody; 2521 alias postJson = postBody; // old name 2522 2523 /* Internal state flags */ 2524 private bool outputtedResponseData; 2525 private bool noCache = true; 2526 2527 const(string[string]) environmentVariables; 2528 2529 /** What follows is data gotten from the HTTP request. It is all fully immutable, 2530 partially because it logically is (your code doesn't change what the user requested...) 2531 and partially because I hate how bad programs in PHP change those superglobals to do 2532 all kinds of hard to follow ugliness. I don't want that to ever happen in D. 2533 2534 For some of these, you'll want to refer to the http or cgi specs for more details. 2535 */ 2536 immutable(string[string]) requestHeaders; /// All the raw headers in the request as name/value pairs. The name is stored as all lower case, but otherwise the same as it is in HTTP; words separated by dashes. For example, "cookie" or "accept-encoding". Many HTTP headers have specialized variables below for more convenience and static name checking; you should generally try to use them. 2537 2538 immutable(char[]) host; /// The hostname in the request. If one program serves multiple domains, you can use this to differentiate between them. 2539 immutable(char[]) origin; /// The origin header in the request, if present. Some HTML5 cross-domain apis set this and you should check it on those cross domain requests and websockets. 2540 immutable(char[]) userAgent; /// The browser's user-agent string. Can be used to identify the browser. 2541 immutable(char[]) pathInfo; /// This is any stuff sent after your program's name on the url, but before the query string. For example, suppose your program is named "app". If the user goes to site.com/app, pathInfo is empty. But, he can also go to site.com/app/some/sub/path; treating your program like a virtual folder. In this case, pathInfo == "/some/sub/path". 2542 immutable(char[]) scriptName; /// The full base path of your program, as seen by the user. If your program is located at site.com/programs/apps, scriptName == "/programs/apps". 2543 immutable(char[]) scriptFileName; /// The physical filename of your script 2544 immutable(char[]) authorization; /// The full authorization string from the header, undigested. Useful for implementing auth schemes such as OAuth 1.0. Note that some web servers do not forward this to the app without taking extra steps. See requireBasicAuth's comment for more info. 2545 immutable(char[]) accept; /// The HTTP accept header is the user agent telling what content types it is willing to accept. This is often */*; they accept everything, so it's not terribly useful. (The similar sounding Accept-Encoding header is handled automatically for chunking and gzipping. Simply set gzipResponse = true and cgi.d handles the details, zipping if the user's browser is willing to accept it.) 2546 immutable(char[]) lastEventId; /// The HTML 5 draft includes an EventSource() object that connects to the server, and remains open to take a stream of events. My arsd.rtud module can help with the server side part of that. The Last-Event-Id http header is defined in the draft to help handle loss of connection. When the browser reconnects to you, it sets this header to the last event id it saw, so you can catch it up. This member has the contents of that header. 2547 2548 immutable(RequestMethod) requestMethod; /// The HTTP request verb: GET, POST, etc. It is represented as an enum in cgi.d (which, like many enums, you can convert back to string with std.conv.to()). A HTTP GET is supposed to, according to the spec, not have side effects; a user can GET something over and over again and always have the same result. On all requests, the get[] and getArray[] members may be filled in. The post[] and postArray[] members are only filled in on POST methods. 2549 immutable(char[]) queryString; /// The unparsed content of the request query string - the stuff after the ? in your URL. See get[] and getArray[] for a parse view of it. Sometimes, the unparsed string is useful though if you want a custom format of data up there (probably not a good idea, unless it is really simple, like "?username" perhaps.) 2550 immutable(char[]) cookie; /// The unparsed content of the Cookie: header in the request. See also the cookies[string] member for a parsed view of the data. 2551 /** The Referer header from the request. (It is misspelled in the HTTP spec, and thus the actual request and cgi specs too, but I spelled the word correctly here because that's sane. The spec's misspelling is an implementation detail.) It contains the site url that referred the user to your program; the site that linked to you, or if you're serving images, the site that has you as an image. Also, if you're in an iframe, the referrer is the site that is framing you. 2552 2553 Important note: if the user copy/pastes your url, this is blank, and, just like with all other user data, their browsers can also lie to you. Don't rely on it for real security. 2554 */ 2555 immutable(char[]) referrer; 2556 immutable(char[]) requestUri; /// The full url if the current request, excluding the protocol and host. requestUri == scriptName ~ pathInfo ~ (queryString.length ? "?" ~ queryString : ""); 2557 2558 immutable(char[]) remoteAddress; /// The IP address of the user, as we see it. (Might not match the IP of the user's computer due to things like proxies and NAT.) 2559 2560 immutable bool https; /// Was the request encrypted via https? 2561 immutable int port; /// On what TCP port number did the server receive the request? 2562 2563 /** Here come the parsed request variables - the things that come close to PHP's _GET, _POST, etc. superglobals in content. */ 2564 2565 immutable(string[string]) get; /// The data from your query string in the url, only showing the last string of each name. If you want to handle multiple values with the same name, use getArray. This only works right if the query string is x-www-form-urlencoded; the default you see on the web with name=value pairs separated by the & character. 2566 immutable(string[string]) post; /// The data from the request's body, on POST requests. It parses application/x-www-form-urlencoded data (used by most web requests, including typical forms), and multipart/form-data requests (used by file uploads on web forms) into the same container, so you can always access them the same way. It makes no attempt to parse other content types. If you want to accept an XML Post body (for a web api perhaps), you'll need to handle the raw data yourself. 2567 immutable(string[string]) cookies; /// Separates out the cookie header into individual name/value pairs (which is how you set them!) 2568 2569 /** 2570 Represents user uploaded files. 2571 2572 When making a file upload form, be sure to follow the standard: set method="POST" and enctype="multipart/form-data" in your html <form> tag attributes. The key into this array is the name attribute on your input tag, just like with other post variables. See the comments on the UploadedFile struct for more information about the data inside, including important notes on max size and content location. 2573 */ 2574 immutable(UploadedFile[][string]) filesArray; 2575 immutable(UploadedFile[string]) files; 2576 2577 /// Use these if you expect multiple items submitted with the same name. btw, assert(get[name] is getArray[name][$-1); should pass. Same for post and cookies. 2578 /// the order of the arrays is the order the data arrives 2579 immutable(string[][string]) getArray; /// like get, but an array of values per name 2580 immutable(string[][string]) postArray; /// ditto for post 2581 immutable(string[][string]) cookiesArray; /// ditto for cookies 2582 2583 // convenience function for appending to a uri without extra ? 2584 // matches the name and effect of javascript's location.search property 2585 string search() const { 2586 if(queryString.length) 2587 return "?" ~ queryString; 2588 return ""; 2589 } 2590 2591 // FIXME: what about multiple files with the same name? 2592 private: 2593 //RequestMethod _requestMethod; 2594 } 2595 2596 /// use this for testing or other isolated things when you want it to be no-ops 2597 Cgi dummyCgi(Cgi.RequestMethod method = Cgi.RequestMethod.GET, string url = null, in ubyte[] data = null, void delegate(const(ubyte)[]) outputSink = null) { 2598 // we want to ignore, not use stdout 2599 if(outputSink is null) 2600 outputSink = delegate void(const(ubyte)[]) { }; 2601 2602 string[string] env; 2603 env["REQUEST_METHOD"] = to!string(method); 2604 env["CONTENT_LENGTH"] = to!string(data.length); 2605 2606 auto cgi = new Cgi( 2607 0, 2608 env, 2609 { return data; }, 2610 outputSink, 2611 null); 2612 2613 return cgi; 2614 } 2615 2616 /++ 2617 A helper test class for request handler unittests. 2618 +/ 2619 version(with_breaking_cgi_features) 2620 class CgiTester { 2621 private { 2622 SessionObject[TypeInfo] mockSessions; 2623 SessionObject getSessionOverride(TypeInfo ti) { 2624 if(auto o = ti in mockSessions) 2625 return *o; 2626 else 2627 return null; 2628 } 2629 void setSessionOverride(TypeInfo ti, SessionObject so) { 2630 mockSessions[ti] = so; 2631 } 2632 } 2633 2634 /++ 2635 Gets (and creates if necessary) a mock session object for this test. Note 2636 it will be the same one used for any test operations through this CgiTester instance. 2637 +/ 2638 Session!Data getSessionObject(Data)() { 2639 auto obj = getSessionOverride(typeid(typeof(return))); 2640 if(obj !is null) 2641 return cast(typeof(return)) obj; 2642 else { 2643 auto o = new MockSession!Data(); 2644 setSessionOverride(typeid(typeof(return)), o); 2645 return o; 2646 } 2647 } 2648 2649 /++ 2650 Pass a reference to your request handler when creating the tester. 2651 +/ 2652 this(void function(Cgi) requestHandler) { 2653 this.requestHandler = requestHandler; 2654 } 2655 2656 /++ 2657 You can check response information with these methods after you call the request handler. 2658 +/ 2659 struct Response { 2660 int code; 2661 string[string] headers; 2662 string responseText; 2663 ubyte[] responseBody; 2664 } 2665 2666 /++ 2667 Executes a test request on your request handler, and returns the response. 2668 2669 Params: 2670 url = The URL to test. Should be an absolute path, but excluding domain. e.g. `"/test"`. 2671 args = additional arguments. Same format as cgi's command line handler. 2672 +/ 2673 Response GET(string url, string[] args = null) { 2674 return executeTest("GET", url, args); 2675 } 2676 /// ditto 2677 Response POST(string url, string[] args = null) { 2678 return executeTest("POST", url, args); 2679 } 2680 2681 /// ditto 2682 Response executeTest(string method, string url, string[] args) { 2683 ubyte[] outputtedRawData; 2684 void outputSink(const(ubyte)[] data) { 2685 outputtedRawData ~= data; 2686 } 2687 auto cgi = new Cgi(["test", method, url] ~ args, &outputSink); 2688 cgi.testInProcess = this; 2689 scope(exit) cgi.dispose(); 2690 2691 requestHandler(cgi); 2692 2693 cgi.close(); 2694 2695 Response response; 2696 2697 if(outputtedRawData.length) { 2698 enum LINE = "\r\n"; 2699 2700 auto idx = outputtedRawData.locationOf(LINE ~ LINE); 2701 assert(idx != -1, to!string(outputtedRawData)); 2702 auto headers = cast(string) outputtedRawData[0 .. idx]; 2703 response.code = 200; 2704 while(headers.length) { 2705 auto i = headers.locationOf(LINE); 2706 if(i == -1) i = cast(int) headers.length; 2707 2708 auto header = headers[0 .. i]; 2709 2710 auto c = header.locationOf(":"); 2711 if(c != -1) { 2712 auto name = header[0 .. c]; 2713 auto value = header[c + 2 ..$]; 2714 2715 if(name == "Status") 2716 response.code = value[0 .. value.locationOf(" ")].to!int; 2717 2718 response.headers[name] = value; 2719 } else { 2720 assert(0); 2721 } 2722 2723 if(i != headers.length) 2724 i += 2; 2725 headers = headers[i .. $]; 2726 } 2727 response.responseBody = outputtedRawData[idx + 4 .. $]; 2728 response.responseText = cast(string) response.responseBody; 2729 } 2730 2731 return response; 2732 } 2733 2734 private void function(Cgi) requestHandler; 2735 } 2736 2737 2738 // should this be a separate module? Probably, but that's a hassle. 2739 2740 /// Makes a data:// uri that can be used as links in most newer browsers (IE8+). 2741 string makeDataUrl(string mimeType, in void[] data) { 2742 auto data64 = Base64.encode(cast(const(ubyte[])) data); 2743 return "data:" ~ mimeType ~ ";base64," ~ assumeUnique(data64); 2744 } 2745 2746 // FIXME: I don't think this class correctly decodes/encodes the individual parts 2747 /// Represents a url that can be broken down or built up through properties 2748 struct Uri { 2749 alias toString this; // blargh idk a url really is a string, but should it be implicit? 2750 2751 // scheme//userinfo@host:port/path?query#fragment 2752 2753 string scheme; /// e.g. "http" in "http://example.com/" 2754 string userinfo; /// the username (and possibly a password) in the uri 2755 string host; /// the domain name 2756 int port; /// port number, if given. Will be zero if a port was not explicitly given 2757 string path; /// e.g. "/folder/file.html" in "http://example.com/folder/file.html" 2758 string query; /// the stuff after the ? in a uri 2759 string fragment; /// the stuff after the # in a uri. 2760 2761 // idk if i want to keep these, since the functions they wrap are used many, many, many times in existing code, so this is either an unnecessary alias or a gratuitous break of compatibility 2762 // the decode ones need to keep different names anyway because we can't overload on return values... 2763 static string encode(string s) { return std.uri.encodeComponent(s); } 2764 static string encode(string[string] s) { return encodeVariables(s); } 2765 static string encode(string[][string] s) { return encodeVariables(s); } 2766 2767 /// Breaks down a uri string to its components 2768 this(string uri) { 2769 reparse(uri); 2770 } 2771 2772 private void reparse(string uri) { 2773 // from RFC 3986 2774 // the ctRegex triples the compile time and makes ugly errors for no real benefit 2775 // it was a nice experiment but just not worth it. 2776 // enum ctr = ctRegex!r"^(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\?([^#]*))?(#(.*))?"; 2777 /* 2778 Captures: 2779 0 = whole url 2780 1 = scheme, with : 2781 2 = scheme, no : 2782 3 = authority, with // 2783 4 = authority, no // 2784 5 = path 2785 6 = query string, with ? 2786 7 = query string, no ? 2787 8 = anchor, with # 2788 9 = anchor, no # 2789 */ 2790 // Yikes, even regular, non-CT regex is also unacceptably slow to compile. 1.9s on my computer! 2791 // instead, I will DIY and cut that down to 0.6s on the same computer. 2792 /* 2793 2794 Note that authority is 2795 user:password@domain:port 2796 where the user:password@ part is optional, and the :port is optional. 2797 2798 Regex translation: 2799 2800 Scheme cannot have :, /, ?, or # in it, and must have one or more chars and end in a :. It is optional, but must be first. 2801 Authority must start with //, but cannot have any other /, ?, or # in it. It is optional. 2802 Path cannot have any ? or # in it. It is optional. 2803 Query must start with ? and must not have # in it. It is optional. 2804 Anchor must start with # and can have anything else in it to end of string. It is optional. 2805 */ 2806 2807 this = Uri.init; // reset all state 2808 2809 // empty uri = nothing special 2810 if(uri.length == 0) { 2811 return; 2812 } 2813 2814 size_t idx; 2815 2816 scheme_loop: foreach(char c; uri[idx .. $]) { 2817 switch(c) { 2818 case ':': 2819 case '/': 2820 case '?': 2821 case '#': 2822 break scheme_loop; 2823 default: 2824 } 2825 idx++; 2826 } 2827 2828 if(idx == 0 && uri[idx] == ':') { 2829 // this is actually a path! we skip way ahead 2830 goto path_loop; 2831 } 2832 2833 if(idx == uri.length) { 2834 // the whole thing is a path, apparently 2835 path = uri; 2836 return; 2837 } 2838 2839 if(idx > 0 && uri[idx] == ':') { 2840 scheme = uri[0 .. idx]; 2841 idx++; 2842 } else { 2843 // we need to rewind; it found a / but no :, so the whole thing is prolly a path... 2844 idx = 0; 2845 } 2846 2847 if(idx + 2 < uri.length && uri[idx .. idx + 2] == "//") { 2848 // we have an authority.... 2849 idx += 2; 2850 2851 auto authority_start = idx; 2852 authority_loop: foreach(char c; uri[idx .. $]) { 2853 switch(c) { 2854 case '/': 2855 case '?': 2856 case '#': 2857 break authority_loop; 2858 default: 2859 } 2860 idx++; 2861 } 2862 2863 auto authority = uri[authority_start .. idx]; 2864 2865 auto idx2 = authority.indexOf("@"); 2866 if(idx2 != -1) { 2867 userinfo = authority[0 .. idx2]; 2868 authority = authority[idx2 + 1 .. $]; 2869 } 2870 2871 if(authority.length && authority[0] == '[') { 2872 // ipv6 address special casing 2873 idx2 = authority.indexOf(']'); 2874 if(idx2 != -1) { 2875 auto end = authority[idx2 + 1 .. $]; 2876 if(end.length && end[0] == ':') 2877 idx2 = idx2 + 1; 2878 else 2879 idx2 = -1; 2880 } 2881 } else { 2882 idx2 = authority.indexOf(":"); 2883 } 2884 2885 if(idx2 == -1) { 2886 port = 0; // 0 means not specified; we should use the default for the scheme 2887 host = authority; 2888 } else { 2889 host = authority[0 .. idx2]; 2890 port = to!int(authority[idx2 + 1 .. $]); 2891 } 2892 } 2893 2894 path_loop: 2895 auto path_start = idx; 2896 2897 foreach(char c; uri[idx .. $]) { 2898 if(c == '?' || c == '#') 2899 break; 2900 idx++; 2901 } 2902 2903 path = uri[path_start .. idx]; 2904 2905 if(idx == uri.length) 2906 return; // nothing more to examine... 2907 2908 if(uri[idx] == '?') { 2909 idx++; 2910 auto query_start = idx; 2911 foreach(char c; uri[idx .. $]) { 2912 if(c == '#') 2913 break; 2914 idx++; 2915 } 2916 query = uri[query_start .. idx]; 2917 } 2918 2919 if(idx < uri.length && uri[idx] == '#') { 2920 idx++; 2921 fragment = uri[idx .. $]; 2922 } 2923 2924 // uriInvalidated = false; 2925 } 2926 2927 private string rebuildUri() const { 2928 string ret; 2929 if(scheme.length) 2930 ret ~= scheme ~ ":"; 2931 if(userinfo.length || host.length) 2932 ret ~= "//"; 2933 if(userinfo.length) 2934 ret ~= userinfo ~ "@"; 2935 if(host.length) 2936 ret ~= host; 2937 if(port) 2938 ret ~= ":" ~ to!string(port); 2939 2940 ret ~= path; 2941 2942 if(query.length) 2943 ret ~= "?" ~ query; 2944 2945 if(fragment.length) 2946 ret ~= "#" ~ fragment; 2947 2948 // uri = ret; 2949 // uriInvalidated = false; 2950 return ret; 2951 } 2952 2953 /// Converts the broken down parts back into a complete string 2954 string toString() const { 2955 // if(uriInvalidated) 2956 return rebuildUri(); 2957 } 2958 2959 /// Returns a new absolute Uri given a base. It treats this one as 2960 /// relative where possible, but absolute if not. (If protocol, domain, or 2961 /// other info is not set, the new one inherits it from the base.) 2962 /// 2963 /// Browsers use a function like this to figure out links in html. 2964 Uri basedOn(in Uri baseUrl) const { 2965 Uri n = this; // copies 2966 // n.uriInvalidated = true; // make sure we regenerate... 2967 2968 // userinfo is not inherited... is this wrong? 2969 2970 // if anything is given in the existing url, we don't use the base anymore. 2971 if(n.scheme.empty) { 2972 n.scheme = baseUrl.scheme; 2973 if(n.host.empty) { 2974 n.host = baseUrl.host; 2975 if(n.port == 0) { 2976 n.port = baseUrl.port; 2977 if(n.path.length > 0 && n.path[0] != '/') { 2978 auto b = baseUrl.path[0 .. baseUrl.path.lastIndexOf("/") + 1]; 2979 if(b.length == 0) 2980 b = "/"; 2981 n.path = b ~ n.path; 2982 } else if(n.path.length == 0) { 2983 n.path = baseUrl.path; 2984 } 2985 } 2986 } 2987 } 2988 2989 n.removeDots(); 2990 2991 return n; 2992 } 2993 2994 void removeDots() { 2995 auto parts = this.path.split("/"); 2996 string[] toKeep; 2997 foreach(part; parts) { 2998 if(part == ".") { 2999 continue; 3000 } else if(part == "..") { 3001 //if(toKeep.length > 1) 3002 toKeep = toKeep[0 .. $-1]; 3003 //else 3004 //toKeep = [""]; 3005 continue; 3006 } else { 3007 //if(toKeep.length && toKeep[$-1].length == 0 && part.length == 0) 3008 //continue; // skip a `//` situation 3009 toKeep ~= part; 3010 } 3011 } 3012 3013 auto path = toKeep.join("/"); 3014 if(path.length && path[0] != '/') 3015 path = "/" ~ path; 3016 3017 this.path = path; 3018 } 3019 3020 unittest { 3021 auto uri = Uri("test.html"); 3022 assert(uri.path == "test.html"); 3023 uri = Uri("path/1/lol"); 3024 assert(uri.path == "path/1/lol"); 3025 uri = Uri("http://me@example.com"); 3026 assert(uri.scheme == "http"); 3027 assert(uri.userinfo == "me"); 3028 assert(uri.host == "example.com"); 3029 uri = Uri("http://example.com/#a"); 3030 assert(uri.scheme == "http"); 3031 assert(uri.host == "example.com"); 3032 assert(uri.fragment == "a"); 3033 uri = Uri("#foo"); 3034 assert(uri.fragment == "foo"); 3035 uri = Uri("?lol"); 3036 assert(uri.query == "lol"); 3037 uri = Uri("#foo?lol"); 3038 assert(uri.fragment == "foo?lol"); 3039 uri = Uri("?lol#foo"); 3040 assert(uri.fragment == "foo"); 3041 assert(uri.query == "lol"); 3042 3043 uri = Uri("http://127.0.0.1/"); 3044 assert(uri.host == "127.0.0.1"); 3045 assert(uri.port == 0); 3046 3047 uri = Uri("http://127.0.0.1:123/"); 3048 assert(uri.host == "127.0.0.1"); 3049 assert(uri.port == 123); 3050 3051 uri = Uri("http://[ff:ff::0]/"); 3052 assert(uri.host == "[ff:ff::0]"); 3053 3054 uri = Uri("http://[ff:ff::0]:123/"); 3055 assert(uri.host == "[ff:ff::0]"); 3056 assert(uri.port == 123); 3057 } 3058 3059 // This can sometimes be a big pain in the butt for me, so lots of copy/paste here to cover 3060 // the possibilities. 3061 unittest { 3062 auto url = Uri("cool.html"); // checking relative links 3063 3064 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/what/cool.html"); 3065 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/what/cool.html"); 3066 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/what/cool.html"); 3067 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/cool.html"); 3068 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/cool.html"); 3069 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/what/cool.html"); 3070 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/what/cool.html"); 3071 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/what/cool.html"); 3072 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/cool.html"); 3073 3074 url = Uri("/something/cool.html"); // same server, different path 3075 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/something/cool.html"); 3076 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/something/cool.html"); 3077 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/something/cool.html"); 3078 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/something/cool.html"); 3079 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/something/cool.html"); 3080 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/something/cool.html"); 3081 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/something/cool.html"); 3082 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/something/cool.html"); 3083 assert(url.basedOn(Uri("http://test.com")) == "http://test.com/something/cool.html"); 3084 3085 url = Uri("?query=answer"); // same path. server, protocol, and port, just different query string and fragment 3086 assert(url.basedOn(Uri("http://test.com/what/test.html")) == "http://test.com/what/test.html?query=answer"); 3087 assert(url.basedOn(Uri("https://test.com/what/test.html")) == "https://test.com/what/test.html?query=answer"); 3088 assert(url.basedOn(Uri("http://test.com/what/")) == "http://test.com/what/?query=answer"); 3089 assert(url.basedOn(Uri("http://test.com/")) == "http://test.com/?query=answer"); 3090 assert(url.basedOn(Uri("http://test.com")) == "http://test.com?query=answer"); 3091 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b")) == "http://test.com/what/test.html?query=answer"); 3092 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d")) == "http://test.com/what/test.html?query=answer"); 3093 assert(url.basedOn(Uri("http://test.com/what/test.html?a=b&c=d#what")) == "http://test.com/what/test.html?query=answer"); 3094 assert(url.basedOn(Uri("http://test.com")) == "http://test.com?query=answer"); 3095 3096 url = Uri("/test/bar"); 3097 assert(Uri("./").basedOn(url) == "/test/", Uri("./").basedOn(url)); 3098 assert(Uri("../").basedOn(url) == "/"); 3099 3100 url = Uri("http://example.com/"); 3101 assert(Uri("../foo").basedOn(url) == "http://example.com/foo"); 3102 3103 //auto uriBefore = url; 3104 url = Uri("#anchor"); // everything should remain the same except the anchor 3105 //uriBefore.anchor = "anchor"); 3106 //assert(url == uriBefore); 3107 3108 url = Uri("//example.com"); // same protocol, but different server. the path here should be blank. 3109 3110 url = Uri("//example.com/example.html"); // same protocol, but different server and path 3111 3112 url = Uri("http://example.com/test.html"); // completely absolute link should never be modified 3113 3114 url = Uri("http://example.com"); // completely absolute link should never be modified, even if it has no path 3115 3116 // FIXME: add something for port too 3117 } 3118 3119 // these are like javascript's location.search and location.hash 3120 string search() const { 3121 return query.length ? ("?" ~ query) : ""; 3122 } 3123 string hash() const { 3124 return fragment.length ? ("#" ~ fragment) : ""; 3125 } 3126 } 3127 3128 3129 /* 3130 for session, see web.d 3131 */ 3132 3133 /// breaks down a url encoded string 3134 string[][string] decodeVariables(string data, string separator = "&", string[]* namesInOrder = null, string[]* valuesInOrder = null) { 3135 auto vars = data.split(separator); 3136 string[][string] _get; 3137 foreach(var; vars) { 3138 auto equal = var.indexOf("="); 3139 string name; 3140 string value; 3141 if(equal == -1) { 3142 name = decodeComponent(var); 3143 value = ""; 3144 } else { 3145 //_get[decodeComponent(var[0..equal])] ~= decodeComponent(var[equal + 1 .. $].replace("+", " ")); 3146 // stupid + -> space conversion. 3147 name = decodeComponent(var[0..equal].replace("+", " ")); 3148 value = decodeComponent(var[equal + 1 .. $].replace("+", " ")); 3149 } 3150 3151 _get[name] ~= value; 3152 if(namesInOrder) 3153 (*namesInOrder) ~= name; 3154 if(valuesInOrder) 3155 (*valuesInOrder) ~= value; 3156 } 3157 return _get; 3158 } 3159 3160 /// breaks down a url encoded string, but only returns the last value of any array 3161 string[string] decodeVariablesSingle(string data) { 3162 string[string] va; 3163 auto varArray = decodeVariables(data); 3164 foreach(k, v; varArray) 3165 va[k] = v[$-1]; 3166 3167 return va; 3168 } 3169 3170 /// url encodes the whole string 3171 string encodeVariables(in string[string] data) { 3172 string ret; 3173 3174 bool outputted = false; 3175 foreach(k, v; data) { 3176 if(outputted) 3177 ret ~= "&"; 3178 else 3179 outputted = true; 3180 3181 ret ~= std.uri.encodeComponent(k) ~ "=" ~ std.uri.encodeComponent(v); 3182 } 3183 3184 return ret; 3185 } 3186 3187 /// url encodes a whole string 3188 string encodeVariables(in string[][string] data) { 3189 string ret; 3190 3191 bool outputted = false; 3192 foreach(k, arr; data) { 3193 foreach(v; arr) { 3194 if(outputted) 3195 ret ~= "&"; 3196 else 3197 outputted = true; 3198 ret ~= std.uri.encodeComponent(k) ~ "=" ~ std.uri.encodeComponent(v); 3199 } 3200 } 3201 3202 return ret; 3203 } 3204 3205 /// Encodes all but the explicitly unreserved characters per rfc 3986 3206 /// Alphanumeric and -_.~ are the only ones left unencoded 3207 /// name is borrowed from php 3208 string rawurlencode(in char[] data) { 3209 string ret; 3210 ret.reserve(data.length * 2); 3211 foreach(char c; data) { 3212 if( 3213 (c >= 'a' && c <= 'z') || 3214 (c >= 'A' && c <= 'Z') || 3215 (c >= '0' && c <= '9') || 3216 c == '-' || c == '_' || c == '.' || c == '~') 3217 { 3218 ret ~= c; 3219 } else { 3220 ret ~= '%'; 3221 // since we iterate on char, this should give us the octets of the full utf8 string 3222 ret ~= toHexUpper(c); 3223 } 3224 } 3225 3226 return ret; 3227 } 3228 3229 3230 // http helper functions 3231 3232 // for chunked responses (which embedded http does whenever possible) 3233 version(none) // this is moved up above to avoid making a copy of the data 3234 const(ubyte)[] makeChunk(const(ubyte)[] data) { 3235 const(ubyte)[] ret; 3236 3237 ret = cast(const(ubyte)[]) toHex(data.length); 3238 ret ~= cast(const(ubyte)[]) "\r\n"; 3239 ret ~= data; 3240 ret ~= cast(const(ubyte)[]) "\r\n"; 3241 3242 return ret; 3243 } 3244 3245 string toHex(long num) { 3246 string ret; 3247 while(num) { 3248 int v = num % 16; 3249 num /= 16; 3250 char d = cast(char) ((v < 10) ? v + '0' : (v-10) + 'a'); 3251 ret ~= d; 3252 } 3253 3254 return to!string(array(ret.retro)); 3255 } 3256 3257 string toHexUpper(long num) { 3258 string ret; 3259 while(num) { 3260 int v = num % 16; 3261 num /= 16; 3262 char d = cast(char) ((v < 10) ? v + '0' : (v-10) + 'A'); 3263 ret ~= d; 3264 } 3265 3266 if(ret.length == 1) 3267 ret ~= "0"; // url encoding requires two digits and that's what this function is used for... 3268 3269 return to!string(array(ret.retro)); 3270 } 3271 3272 3273 // the generic mixins 3274 3275 /// Use this instead of writing your own main 3276 mixin template GenericMain(alias fun, long maxContentLength = defaultMaxContentLength) { 3277 mixin CustomCgiMain!(Cgi, fun, maxContentLength); 3278 } 3279 3280 private string simpleHtmlEncode(string s) { 3281 return s.replace("&", "&").replace("<", "<").replace(">", ">").replace("\n", "<br />\n"); 3282 } 3283 3284 string messageFromException(Throwable t) { 3285 string message; 3286 if(t !is null) { 3287 debug message = t.toString(); 3288 else message = "An unexpected error has occurred."; 3289 } else { 3290 message = "Unknown error"; 3291 } 3292 return message; 3293 } 3294 3295 string plainHttpError(bool isCgi, string type, Throwable t) { 3296 auto message = messageFromException(t); 3297 message = simpleHtmlEncode(message); 3298 3299 return format("%s %s\r\nContent-Length: %s\r\n\r\n%s", 3300 isCgi ? "Status:" : "HTTP/1.0", 3301 type, message.length, message); 3302 } 3303 3304 // returns true if we were able to recover reasonably 3305 bool handleException(Cgi cgi, Throwable t) { 3306 if(cgi.isClosed) { 3307 // if the channel has been explicitly closed, we can't handle it here 3308 return true; 3309 } 3310 3311 if(cgi.outputtedResponseData) { 3312 // the headers are sent, but the channel is open... since it closes if all was sent, we can append an error message here. 3313 return false; // but I don't want to, since I don't know what condition the output is in; I don't want to inject something (nor check the content-type for that matter. So we say it was not a clean handling. 3314 } else { 3315 // no headers are sent, we can send a full blown error and recover 3316 cgi.setCache(false); 3317 cgi.setResponseContentType("text/html"); 3318 cgi.setResponseLocation(null); // cancel the redirect 3319 cgi.setResponseStatus("500 Internal Server Error"); 3320 cgi.write(simpleHtmlEncode(messageFromException(t))); 3321 cgi.close(); 3322 return true; 3323 } 3324 } 3325 3326 bool isCgiRequestMethod(string s) { 3327 s = s.toUpper(); 3328 if(s == "COMMANDLINE") 3329 return true; 3330 foreach(member; __traits(allMembers, Cgi.RequestMethod)) 3331 if(s == member) 3332 return true; 3333 return false; 3334 } 3335 3336 /// If you want to use a subclass of Cgi with generic main, use this mixin. 3337 mixin template CustomCgiMain(CustomCgi, alias fun, long maxContentLength = defaultMaxContentLength) if(is(CustomCgi : Cgi)) { 3338 // kinda hacky - the T... is passed to Cgi's constructor in standard cgi mode, and ignored elsewhere 3339 void main(string[] args) { 3340 cgiMainImpl!(fun, CustomCgi, maxContentLength)(args); 3341 } 3342 } 3343 3344 version(embedded_httpd_processes) 3345 __gshared int processPoolSize = 8; 3346 3347 // Returns true if run. You should exit the program after that. 3348 bool tryAddonServers(string[] args) { 3349 if(args.length > 1) { 3350 // run the special separate processes if needed 3351 switch(args[1]) { 3352 case "--websocket-server": 3353 version(with_addon_servers) 3354 websocketServers[args[2]](args[3 .. $]); 3355 else 3356 printf("Add-on servers not compiled in.\n"); 3357 return true; 3358 case "--websocket-servers": 3359 import core.demangle; 3360 version(with_addon_servers_connections) 3361 foreach(k, v; websocketServers) 3362 writeln(k, "\t", demangle(k)); 3363 return true; 3364 case "--session-server": 3365 version(with_addon_servers) 3366 runSessionServer(); 3367 else 3368 printf("Add-on servers not compiled in.\n"); 3369 return true; 3370 case "--event-server": 3371 version(with_addon_servers) 3372 runEventServer(); 3373 else 3374 printf("Add-on servers not compiled in.\n"); 3375 return true; 3376 case "--timer-server": 3377 version(with_addon_servers) 3378 runTimerServer(); 3379 else 3380 printf("Add-on servers not compiled in.\n"); 3381 return true; 3382 case "--timed-jobs": 3383 import core.demangle; 3384 version(with_addon_servers_connections) 3385 foreach(k, v; scheduledJobHandlers) 3386 writeln(k, "\t", demangle(k)); 3387 return true; 3388 case "--timed-job": 3389 scheduledJobHandlers[args[2]](args[3 .. $]); 3390 return true; 3391 default: 3392 // intentionally blank - do nothing and carry on to run normally 3393 } 3394 } 3395 return false; 3396 } 3397 3398 /// Tries to simulate a request from the command line. Returns true if it does, false if it didn't find the args. 3399 bool trySimulatedRequest(alias fun, CustomCgi = Cgi)(string[] args) if(is(CustomCgi : Cgi)) { 3400 // we support command line thing for easy testing everywhere 3401 // it needs to be called ./app method uri [other args...] 3402 if(args.length >= 3 && isCgiRequestMethod(args[1])) { 3403 Cgi cgi = new CustomCgi(args); 3404 scope(exit) cgi.dispose(); 3405 fun(cgi); 3406 cgi.close(); 3407 return true; 3408 } 3409 return false; 3410 } 3411 3412 /++ 3413 A server control and configuration struct, as a potential alternative to calling [GenericMain] or [cgiMainImpl]. See the source of [cgiMainImpl] to an example of how you can use it. 3414 3415 History: 3416 Added Sept 26, 2020 (release version 8.5). 3417 +/ 3418 struct RequestServer { 3419 /// 3420 string listeningHost = defaultListeningHost(); 3421 /// 3422 ushort listeningPort = defaultListeningPort(); 3423 3424 /// 3425 this(string defaultHost, ushort defaultPort) { 3426 this.listeningHost = defaultHost; 3427 this.listeningPort = defaultPort; 3428 } 3429 3430 /// 3431 this(ushort defaultPort) { 3432 listeningPort = defaultPort; 3433 } 3434 3435 /// Reads the args into the other values. 3436 void configureFromCommandLine(string[] args) { 3437 bool foundPort = false; 3438 bool foundHost = false; 3439 bool foundUid = false; 3440 bool foundGid = false; 3441 foreach(arg; args) { 3442 if(foundPort) { 3443 listeningPort = to!ushort(arg); 3444 foundPort = false; 3445 } 3446 if(foundHost) { 3447 listeningHost = arg; 3448 foundHost = false; 3449 } 3450 if(foundUid) { 3451 privDropUserId = to!int(arg); 3452 foundUid = false; 3453 } 3454 if(foundGid) { 3455 privDropGroupId = to!int(arg); 3456 foundGid = false; 3457 } 3458 if(arg == "--listening-host" || arg == "-h" || arg == "/listening-host") 3459 foundHost = true; 3460 else if(arg == "--port" || arg == "-p" || arg == "/port" || arg == "--listening-port") 3461 foundPort = true; 3462 else if(arg == "--uid") 3463 foundUid = true; 3464 else if(arg == "--gid") 3465 foundGid = true; 3466 } 3467 } 3468 3469 // FIXME: the privDropUserId/group id need to be set in here instead of global 3470 3471 /++ 3472 Serves a single HTTP request on this thread, with an embedded server, then stops. Designed for cases like embedded oauth responders 3473 3474 History: 3475 Added Oct 10, 2020. 3476 Example: 3477 3478 --- 3479 import arsd.cgi; 3480 void main() { 3481 RequestServer server = RequestServer("127.0.0.1", 6789); 3482 string oauthCode; 3483 string oauthScope; 3484 server.serveHttpOnce!((cgi) { 3485 oauthCode = cgi.request("code"); 3486 oauthScope = cgi.request("scope"); 3487 cgi.write("Thank you, please return to the application."); 3488 }); 3489 // use the code and scope given 3490 } 3491 --- 3492 +/ 3493 void serveHttpOnce(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3494 import std.socket; 3495 3496 bool tcp; 3497 void delegate() cleanup; 3498 auto socket = startListening(listeningHost, listeningPort, tcp, cleanup, 1); 3499 auto connection = socket.accept(); 3500 doThreadHttpConnectionGuts!(CustomCgi, fun, true)(connection); 3501 3502 if(cleanup) 3503 cleanup(); 3504 } 3505 3506 /++ 3507 Starts serving requests according to the current configuration. 3508 +/ 3509 void serve(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3510 version(netman_httpd) { 3511 // Obsolete! 3512 3513 import arsd.httpd; 3514 // what about forwarding the other constructor args? 3515 // this probably needs a whole redoing... 3516 serveHttp!CustomCgi(&fun, listeningPort);//5005); 3517 return; 3518 } else 3519 version(embedded_httpd_processes) { 3520 serveEmbeddedHttpdProcesses!(fun, CustomCgi)(this); 3521 } else 3522 version(embedded_httpd_threads) { 3523 serveEmbeddedHttp!(fun, CustomCgi, maxContentLength)(); 3524 } else 3525 version(scgi) { 3526 serveScgi!(fun, CustomCgi, maxContentLength)(); 3527 } else 3528 version(fastcgi) { 3529 serveFastCgi!(fun, CustomCgi, maxContentLength)(this); 3530 } else 3531 version(stdio_http) { 3532 serveSingleHttpConnectionOnStdio!(fun, CustomCgi, maxContentLength)(); 3533 } else { 3534 //version=plain_cgi; 3535 handleCgiRequest!(fun, CustomCgi, maxContentLength)(); 3536 } 3537 } 3538 3539 void serveEmbeddedHttp(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3540 auto manager = new ListeningConnectionManager(listeningHost, listeningPort, &doThreadHttpConnection!(CustomCgi, fun)); 3541 manager.listen(); 3542 } 3543 void serveScgi(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3544 auto manager = new ListeningConnectionManager(listeningHost, listeningPort, &doThreadScgiConnection!(CustomCgi, fun, maxContentLength)); 3545 manager.listen(); 3546 } 3547 3548 /++ 3549 Serves a single "connection", but the connection is spoken on stdin and stdout instead of on a socket. 3550 3551 Intended for cases like working from systemd, like discussed here: https://forum.dlang.org/post/avmkfdiitirnrenzljwc@forum.dlang.org 3552 3553 History: 3554 Added May 29, 2021 3555 +/ 3556 void serveSingleHttpConnectionOnStdio(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3557 doThreadHttpConnectionGuts!(CustomCgi, fun, true)(new FakeSocketForStdin()); 3558 } 3559 3560 void stop() { 3561 // FIXME 3562 } 3563 } 3564 3565 private int privDropUserId; 3566 private int privDropGroupId; 3567 3568 // Added Jan 11, 2021 3569 private void dropPrivs() { 3570 version(Posix) { 3571 import core.sys.posix.unistd; 3572 3573 auto userId = privDropUserId; 3574 auto groupId = privDropGroupId; 3575 3576 if((userId != 0 || groupId != 0) && getuid() == 0) { 3577 if(groupId) 3578 setgid(groupId); 3579 if(userId) 3580 setuid(userId); 3581 } 3582 3583 } 3584 // FIXME: Windows? 3585 } 3586 3587 version(embedded_httpd_processes) 3588 void serveEmbeddedHttpdProcesses(alias fun, CustomCgi = Cgi)(RequestServer params) { 3589 import core.sys.posix.unistd; 3590 import core.sys.posix.sys.socket; 3591 import core.sys.posix.netinet.in_; 3592 //import std.c.linux.socket; 3593 3594 int sock = socket(AF_INET, SOCK_STREAM, 0); 3595 if(sock == -1) 3596 throw new Exception("socket"); 3597 3598 cloexec(sock); 3599 3600 { 3601 3602 sockaddr_in addr; 3603 addr.sin_family = AF_INET; 3604 addr.sin_port = htons(params.listeningPort); 3605 auto lh = params.listeningHost; 3606 if(lh.length) { 3607 if(inet_pton(AF_INET, lh.toStringz(), &addr.sin_addr.s_addr) != 1) 3608 throw new Exception("bad listening host given, please use an IP address.\nExample: --listening-host 127.0.0.1 means listen only on Localhost.\nExample: --listening-host 0.0.0.0 means listen on all interfaces.\nOr you can pass any other single numeric IPv4 address."); 3609 } else 3610 addr.sin_addr.s_addr = INADDR_ANY; 3611 3612 // HACKISH 3613 int on = 1; 3614 setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &on, on.sizeof); 3615 // end hack 3616 3617 3618 if(bind(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) { 3619 close(sock); 3620 throw new Exception("bind"); 3621 } 3622 3623 // FIXME: if this queue is full, it will just ignore it 3624 // and wait for the client to retransmit it. This is an 3625 // obnoxious timeout condition there. 3626 if(sock.listen(128) == -1) { 3627 close(sock); 3628 throw new Exception("listen"); 3629 } 3630 dropPrivs(); 3631 } 3632 3633 version(embedded_httpd_processes_accept_after_fork) {} else { 3634 int pipeReadFd; 3635 int pipeWriteFd; 3636 3637 { 3638 int[2] pipeFd; 3639 if(socketpair(AF_UNIX, SOCK_DGRAM, 0, pipeFd)) { 3640 import core.stdc.errno; 3641 throw new Exception("pipe failed " ~ to!string(errno)); 3642 } 3643 3644 pipeReadFd = pipeFd[0]; 3645 pipeWriteFd = pipeFd[1]; 3646 } 3647 } 3648 3649 3650 int processCount; 3651 pid_t newPid; 3652 reopen: 3653 while(processCount < processPoolSize) { 3654 newPid = fork(); 3655 if(newPid == 0) { 3656 // start serving on the socket 3657 //ubyte[4096] backingBuffer; 3658 for(;;) { 3659 bool closeConnection; 3660 uint i; 3661 sockaddr addr; 3662 i = addr.sizeof; 3663 version(embedded_httpd_processes_accept_after_fork) { 3664 int s = accept(sock, &addr, &i); 3665 int opt = 1; 3666 import core.sys.posix.netinet.tcp; 3667 // the Cgi class does internal buffering, so disabling this 3668 // helps with latency in many cases... 3669 setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &opt, opt.sizeof); 3670 cloexec(s); 3671 } else { 3672 int s; 3673 auto readret = read_fd(pipeReadFd, &s, s.sizeof, &s); 3674 if(readret != s.sizeof) { 3675 import core.stdc.errno; 3676 throw new Exception("pipe read failed " ~ to!string(errno)); 3677 } 3678 3679 //writeln("process ", getpid(), " got socket ", s); 3680 } 3681 3682 try { 3683 3684 if(s == -1) 3685 throw new Exception("accept"); 3686 3687 scope(failure) close(s); 3688 //ubyte[__traits(classInstanceSize, BufferedInputRange)] bufferedRangeContainer; 3689 auto ir = new BufferedInputRange(s); 3690 //auto ir = emplace!BufferedInputRange(bufferedRangeContainer, s, backingBuffer); 3691 3692 while(!ir.empty) { 3693 //ubyte[__traits(classInstanceSize, CustomCgi)] cgiContainer; 3694 3695 Cgi cgi; 3696 try { 3697 cgi = new CustomCgi(ir, &closeConnection); 3698 cgi._outputFileHandle = s; 3699 // if we have a single process and the browser tries to leave the connection open while concurrently requesting another, it will block everything an deadlock since there's no other server to accept it. By closing after each request in this situation, it tells the browser to serialize for us. 3700 if(processPoolSize <= 1) 3701 closeConnection = true; 3702 //cgi = emplace!CustomCgi(cgiContainer, ir, &closeConnection); 3703 } catch(Throwable t) { 3704 // a construction error is either bad code or bad request; bad request is what it should be since this is bug free :P 3705 // anyway let's kill the connection 3706 version(CRuntime_Musl) { 3707 // LockingTextWriter fails here 3708 // so working around it 3709 auto estr = t.toString(); 3710 stderr.rawWrite(estr); 3711 stderr.rawWrite("\n"); 3712 } else 3713 stderr.writeln(t.toString()); 3714 sendAll(ir.source, plainHttpError(false, "400 Bad Request", t)); 3715 closeConnection = true; 3716 break; 3717 } 3718 assert(cgi !is null); 3719 scope(exit) 3720 cgi.dispose(); 3721 3722 try { 3723 fun(cgi); 3724 cgi.close(); 3725 if(cgi.websocketMode) 3726 closeConnection = true; 3727 } catch(ConnectionException ce) { 3728 closeConnection = true; 3729 } catch(Throwable t) { 3730 // a processing error can be recovered from 3731 version(CRuntime_Musl) { 3732 // LockingTextWriter fails here 3733 // so working around it 3734 auto estr = t.toString(); 3735 stderr.rawWrite(estr); 3736 } else { 3737 stderr.writeln(t.toString); 3738 } 3739 if(!handleException(cgi, t)) 3740 closeConnection = true; 3741 } 3742 3743 if(closeConnection) { 3744 ir.source.close(); 3745 break; 3746 } else { 3747 if(!ir.empty) 3748 ir.popFront(); // get the next 3749 else if(ir.sourceClosed) { 3750 ir.source.close(); 3751 } 3752 } 3753 } 3754 3755 ir.source.close(); 3756 } catch(Throwable t) { 3757 version(CRuntime_Musl) {} else 3758 debug writeln(t); 3759 // most likely cause is a timeout 3760 } 3761 } 3762 } else if(newPid < 0) { 3763 throw new Exception("fork failed"); 3764 } else { 3765 processCount++; 3766 } 3767 } 3768 3769 // the parent should wait for its children... 3770 if(newPid) { 3771 import core.sys.posix.sys.wait; 3772 3773 version(embedded_httpd_processes_accept_after_fork) {} else { 3774 import core.sys.posix.sys.select; 3775 int[] fdQueue; 3776 while(true) { 3777 // writeln("select call"); 3778 int nfds = pipeWriteFd; 3779 if(sock > pipeWriteFd) 3780 nfds = sock; 3781 nfds += 1; 3782 fd_set read_fds; 3783 fd_set write_fds; 3784 FD_ZERO(&read_fds); 3785 FD_ZERO(&write_fds); 3786 FD_SET(sock, &read_fds); 3787 if(fdQueue.length) 3788 FD_SET(pipeWriteFd, &write_fds); 3789 auto ret = select(nfds, &read_fds, &write_fds, null, null); 3790 if(ret == -1) { 3791 import core.stdc.errno; 3792 if(errno == EINTR) 3793 goto try_wait; 3794 else 3795 throw new Exception("wtf select"); 3796 } 3797 3798 int s = -1; 3799 if(FD_ISSET(sock, &read_fds)) { 3800 uint i; 3801 sockaddr addr; 3802 i = addr.sizeof; 3803 s = accept(sock, &addr, &i); 3804 cloexec(s); 3805 import core.sys.posix.netinet.tcp; 3806 int opt = 1; 3807 setsockopt(s, IPPROTO_TCP, TCP_NODELAY, &opt, opt.sizeof); 3808 } 3809 3810 if(FD_ISSET(pipeWriteFd, &write_fds)) { 3811 if(s == -1 && fdQueue.length) { 3812 s = fdQueue[0]; 3813 fdQueue = fdQueue[1 .. $]; // FIXME reuse buffer 3814 } 3815 write_fd(pipeWriteFd, &s, s.sizeof, s); 3816 close(s); // we are done with it, let the other process take ownership 3817 } else 3818 fdQueue ~= s; 3819 } 3820 } 3821 3822 try_wait: 3823 3824 int status; 3825 while(-1 != wait(&status)) { 3826 version(CRuntime_Musl) {} else { 3827 import std.stdio; writeln("Process died ", status); 3828 } 3829 processCount--; 3830 goto reopen; 3831 } 3832 close(sock); 3833 } 3834 } 3835 3836 version(fastcgi) 3837 void serveFastCgi(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)(RequestServer params) { 3838 // SetHandler fcgid-script 3839 FCGX_Stream* input, output, error; 3840 FCGX_ParamArray env; 3841 3842 3843 3844 const(ubyte)[] getFcgiChunk() { 3845 const(ubyte)[] ret; 3846 while(FCGX_HasSeenEOF(input) != -1) 3847 ret ~= cast(ubyte) FCGX_GetChar(input); 3848 return ret; 3849 } 3850 3851 void writeFcgi(const(ubyte)[] data) { 3852 FCGX_PutStr(data.ptr, data.length, output); 3853 } 3854 3855 void doARequest() { 3856 string[string] fcgienv; 3857 3858 for(auto e = env; e !is null && *e !is null; e++) { 3859 string cur = to!string(*e); 3860 auto idx = cur.indexOf("="); 3861 string name, value; 3862 if(idx == -1) 3863 name = cur; 3864 else { 3865 name = cur[0 .. idx]; 3866 value = cur[idx + 1 .. $]; 3867 } 3868 3869 fcgienv[name] = value; 3870 } 3871 3872 void flushFcgi() { 3873 FCGX_FFlush(output); 3874 } 3875 3876 Cgi cgi; 3877 try { 3878 cgi = new CustomCgi(maxContentLength, fcgienv, &getFcgiChunk, &writeFcgi, &flushFcgi); 3879 } catch(Throwable t) { 3880 FCGX_PutStr(cast(ubyte*) t.msg.ptr, t.msg.length, error); 3881 writeFcgi(cast(const(ubyte)[]) plainHttpError(true, "400 Bad Request", t)); 3882 return; //continue; 3883 } 3884 assert(cgi !is null); 3885 scope(exit) cgi.dispose(); 3886 try { 3887 fun(cgi); 3888 cgi.close(); 3889 } catch(Throwable t) { 3890 // log it to the error stream 3891 FCGX_PutStr(cast(ubyte*) t.msg.ptr, t.msg.length, error); 3892 // handle it for the user, if we can 3893 if(!handleException(cgi, t)) 3894 return; // continue; 3895 } 3896 } 3897 3898 auto lp = params.listeningPort; 3899 3900 FCGX_Request request; 3901 if(lp) { 3902 // if a listening port was specified on the command line, we want to spawn ourself 3903 // (needed for nginx without spawn-fcgi, e.g. on Windows) 3904 FCGX_Init(); 3905 auto sock = FCGX_OpenSocket(toStringz(params.listeningHost ~ ":" ~ to!string(lp)), 12); 3906 if(sock < 0) 3907 throw new Exception("Couldn't listen on the port"); 3908 FCGX_InitRequest(&request, sock, 0); 3909 while(FCGX_Accept_r(&request) >= 0) { 3910 input = request.inStream; 3911 output = request.outStream; 3912 error = request.errStream; 3913 env = request.envp; 3914 doARequest(); 3915 } 3916 } else { 3917 // otherwise, assume the httpd is doing it (the case for Apache, IIS, and Lighttpd) 3918 // using the version with a global variable since we are separate processes anyway 3919 while(FCGX_Accept(&input, &output, &error, &env) >= 0) { 3920 doARequest(); 3921 } 3922 } 3923 } 3924 3925 /// Returns the default listening port for the current cgi configuration. 8085 for embedded httpd, 4000 for scgi, irrelevant for others. 3926 ushort defaultListeningPort() { 3927 version(netman_httpd) 3928 return 8080; 3929 else version(embedded_httpd_processes) 3930 return 8085; 3931 else version(embedded_httpd_threads) 3932 return 8085; 3933 else version(scgi) 3934 return 4000; 3935 else 3936 return 0; 3937 } 3938 3939 /// Default host for listening. 127.0.0.1 for scgi, null (aka all interfaces) for all others. If you want the server directly accessible from other computers on the network, normally use null. If not, 127.0.0.1 is a bit better. Settable with default handlers with --listening-host command line argument. 3940 string defaultListeningHost() { 3941 version(netman_httpd) 3942 return null; 3943 else version(embedded_httpd_processes) 3944 return null; 3945 else version(embedded_httpd_threads) 3946 return null; 3947 else version(scgi) 3948 return "127.0.0.1"; 3949 else 3950 return null; 3951 3952 } 3953 3954 /++ 3955 This is the function [GenericMain] calls. View its source for some simple boilerplate you can copy/paste and modify, or you can call it yourself from your `main`. 3956 3957 Params: 3958 fun = Your request handler 3959 CustomCgi = a subclass of Cgi, if you wise to customize it further 3960 maxContentLength = max POST size you want to allow 3961 args = command-line arguments 3962 3963 History: 3964 Documented Sept 26, 2020. 3965 +/ 3966 void cgiMainImpl(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)(string[] args) if(is(CustomCgi : Cgi)) { 3967 if(tryAddonServers(args)) 3968 return; 3969 3970 if(trySimulatedRequest!(fun, CustomCgi)(args)) 3971 return; 3972 3973 RequestServer server; 3974 // you can change the port here if you like 3975 // server.listeningPort = 9000; 3976 3977 // then call this to let the command line args override your default 3978 server.configureFromCommandLine(args); 3979 3980 // and serve the request(s). 3981 server.serve!(fun, CustomCgi, maxContentLength)(); 3982 } 3983 3984 //version(plain_cgi) 3985 void handleCgiRequest(alias fun, CustomCgi = Cgi, long maxContentLength = defaultMaxContentLength)() { 3986 // standard CGI is the default version 3987 Cgi cgi; 3988 try { 3989 cgi = new CustomCgi(maxContentLength); 3990 version(Posix) 3991 cgi._outputFileHandle = 1; // stdout 3992 else version(Windows) 3993 cgi._outputFileHandle = GetStdHandle(STD_OUTPUT_HANDLE); 3994 else static assert(0); 3995 } catch(Throwable t) { 3996 version(CRuntime_Musl) { 3997 // LockingTextWriter fails here 3998 // so working around it 3999 auto s = t.toString(); 4000 stderr.rawWrite(s); 4001 stdout.rawWrite(plainHttpError(true, "400 Bad Request", t)); 4002 } else { 4003 stderr.writeln(t.msg); 4004 // the real http server will probably handle this; 4005 // most likely, this is a bug in Cgi. But, oh well. 4006 stdout.write(plainHttpError(true, "400 Bad Request", t)); 4007 } 4008 return; 4009 } 4010 assert(cgi !is null); 4011 scope(exit) cgi.dispose(); 4012 4013 try { 4014 fun(cgi); 4015 cgi.close(); 4016 } catch (Throwable t) { 4017 version(CRuntime_Musl) { 4018 // LockingTextWriter fails here 4019 // so working around it 4020 auto s = t.msg; 4021 stderr.rawWrite(s); 4022 } else { 4023 stderr.writeln(t.msg); 4024 } 4025 if(!handleException(cgi, t)) 4026 return; 4027 } 4028 } 4029 4030 /+ 4031 The event loop for embedded_httpd_threads will prolly fiber dispatch 4032 cgi constructors too, so slow posts will not monopolize a worker thread. 4033 4034 May want to provide the worker task system just need to ensure all the fibers 4035 has a big enough stack for real work... would also ideally like to reuse them. 4036 4037 4038 So prolly bir would switch it to nonblocking. If it would block, it epoll 4039 registers one shot with this existing fiber to take it over. 4040 4041 new connection comes in. it picks a fiber off the free list, 4042 or if there is none, it creates a new one. this fiber handles 4043 this connection the whole time. 4044 4045 epoll triggers the fiber when something comes in. it is called by 4046 a random worker thread, it might change at any time. at least during 4047 the constructor. maybe into the main body it will stay tied to a thread 4048 just so TLS stuff doesn't randomly change in the middle. but I could 4049 specify if you yield all bets are off. 4050 4051 when the request is finished, if there's more data buffered, it just 4052 keeps going. if there is no more data buffered, it epoll ctls to 4053 get triggered when more data comes in. all one shot. 4054 4055 when a connection is closed, the fiber returns and is then reset 4056 and added to the free list. if the free list is full, the fiber is 4057 just freed, this means it will balloon to a certain size but not generally 4058 grow beyond that unless the activity keeps going. 4059 4060 256 KB stack i thnk per fiber. 4,000 active fibers per gigabyte of memory. 4061 4062 So the fiber has its own magic methods to read and write. if they would block, it registers 4063 for epoll and yields. when it returns, it read/writes and then returns back normal control. 4064 4065 basically you issue the command and it tells you when it is done 4066 4067 it needs to DEL the epoll thing when it is closed. add it when opened. mod it when anther thing issued 4068 4069 +/ 4070 4071 version(cgi_use_fiber) 4072 class CgiFiber : Fiber { 4073 private void function(Socket) f_handler; 4074 private void f_handler_dg(Socket s) { // to avoid extra allocation w/ function 4075 f_handler(s); 4076 } 4077 this(void function(Socket) handler) { 4078 this.f_handler = handler; 4079 this(&f_handler_dg); 4080 } 4081 4082 this(void delegate(Socket) handler) { 4083 this.handler = handler; 4084 // FIXME: stack size 4085 super(&run); 4086 } 4087 4088 Socket connection; 4089 void delegate(Socket) handler; 4090 4091 void run() { 4092 handler(connection); 4093 } 4094 4095 void delegate() postYield; 4096 4097 private void setPostYield(scope void delegate() py) @nogc { 4098 postYield = cast(void delegate()) py; 4099 } 4100 4101 void proceed() { 4102 call(); 4103 auto py = postYield; 4104 postYield = null; 4105 if(py !is null) 4106 py(); 4107 if(state == State.TERM) { 4108 import core.memory; 4109 GC.removeRoot(cast(void*) this); 4110 } 4111 } 4112 } 4113 4114 void doThreadHttpConnection(CustomCgi, alias fun)(Socket connection) { 4115 assert(connection !is null); 4116 version(cgi_use_fiber) { 4117 auto fiber = new CgiFiber(&doThreadHttpConnectionGuts!(CustomCgi, fun)); 4118 import core.memory; 4119 GC.addRoot(cast(void*) fiber); 4120 fiber.connection = connection; 4121 fiber.proceed(); 4122 } else { 4123 doThreadHttpConnectionGuts!(CustomCgi, fun)(connection); 4124 } 4125 } 4126 4127 void doThreadHttpConnectionGuts(CustomCgi, alias fun, bool alwaysCloseConnection = false)(Socket connection) { 4128 scope(failure) { 4129 // catch all for other errors 4130 sendAll(connection, plainHttpError(false, "500 Internal Server Error", null)); 4131 connection.close(); 4132 } 4133 4134 bool closeConnection = alwaysCloseConnection; 4135 4136 /+ 4137 ubyte[4096] inputBuffer = void; 4138 ubyte[__traits(classInstanceSize, BufferedInputRange)] birBuffer = void; 4139 ubyte[__traits(classInstanceSize, CustomCgi)] cgiBuffer = void; 4140 4141 birBuffer[] = cast(ubyte[]) typeid(BufferedInputRange).initializer()[]; 4142 BufferedInputRange ir = cast(BufferedInputRange) cast(void*) birBuffer.ptr; 4143 ir.__ctor(connection, inputBuffer[], true); 4144 +/ 4145 4146 auto ir = new BufferedInputRange(connection); 4147 4148 while(!ir.empty) { 4149 4150 if(ir.view.length == 0) { 4151 ir.popFront(); 4152 if(ir.sourceClosed) { 4153 connection.close(); 4154 closeConnection = true; 4155 break; 4156 } 4157 } 4158 4159 Cgi cgi; 4160 try { 4161 cgi = new CustomCgi(ir, &closeConnection); 4162 cgi._outputFileHandle = connection.handle; 4163 } catch(ConnectionClosedException ce) { 4164 closeConnection = true; 4165 break; 4166 } catch(ConnectionException ce) { 4167 // broken pipe or something, just abort the connection 4168 closeConnection = true; 4169 break; 4170 } catch(Throwable t) { 4171 // a construction error is either bad code or bad request; bad request is what it should be since this is bug free :P 4172 // anyway let's kill the connection 4173 version(CRuntime_Musl) { 4174 stderr.rawWrite(t.toString()); 4175 stderr.rawWrite("\n"); 4176 } else { 4177 stderr.writeln(t.toString()); 4178 } 4179 sendAll(connection, plainHttpError(false, "400 Bad Request", t)); 4180 closeConnection = true; 4181 break; 4182 } 4183 assert(cgi !is null); 4184 scope(exit) 4185 cgi.dispose(); 4186 4187 try { 4188 fun(cgi); 4189 cgi.close(); 4190 if(cgi.websocketMode) 4191 closeConnection = true; 4192 } catch(ConnectionException ce) { 4193 // broken pipe or something, just abort the connection 4194 closeConnection = true; 4195 } catch(ConnectionClosedException ce) { 4196 // broken pipe or something, just abort the connection 4197 closeConnection = true; 4198 } catch(Throwable t) { 4199 // a processing error can be recovered from 4200 version(CRuntime_Musl) {} else 4201 stderr.writeln(t.toString); 4202 if(!handleException(cgi, t)) 4203 closeConnection = true; 4204 } 4205 4206 if(closeConnection || alwaysCloseConnection) { 4207 connection.shutdown(SocketShutdown.BOTH); 4208 connection.close(); 4209 ir.dispose(); 4210 closeConnection = false; // don't reclose after loop 4211 break; 4212 } else { 4213 if(ir.front.length) { 4214 ir.popFront(); // we can't just discard the buffer, so get the next bit and keep chugging along 4215 } else if(ir.sourceClosed) { 4216 ir.source.shutdown(SocketShutdown.BOTH); 4217 ir.source.close(); 4218 ir.dispose(); 4219 closeConnection = false; 4220 } else { 4221 continue; 4222 // break; // this was for a keepalive experiment 4223 } 4224 } 4225 } 4226 4227 if(closeConnection) { 4228 connection.shutdown(SocketShutdown.BOTH); 4229 connection.close(); 4230 ir.dispose(); 4231 } 4232 4233 // I am otherwise NOT closing it here because the parent thread might still be able to make use of the keep-alive connection! 4234 } 4235 4236 void doThreadScgiConnection(CustomCgi, alias fun, long maxContentLength)(Socket connection) { 4237 // and now we can buffer 4238 scope(failure) 4239 connection.close(); 4240 4241 import al = std.algorithm; 4242 4243 size_t size; 4244 4245 string[string] headers; 4246 4247 auto range = new BufferedInputRange(connection); 4248 more_data: 4249 auto chunk = range.front(); 4250 // waiting for colon for header length 4251 auto idx = indexOf(cast(string) chunk, ':'); 4252 if(idx == -1) { 4253 try { 4254 range.popFront(); 4255 } catch(Exception e) { 4256 // it is just closed, no big deal 4257 connection.close(); 4258 return; 4259 } 4260 goto more_data; 4261 } 4262 4263 size = to!size_t(cast(string) chunk[0 .. idx]); 4264 chunk = range.consume(idx + 1); 4265 // reading headers 4266 if(chunk.length < size) 4267 range.popFront(0, size + 1); 4268 // we are now guaranteed to have enough 4269 chunk = range.front(); 4270 assert(chunk.length > size); 4271 4272 idx = 0; 4273 string key; 4274 string value; 4275 foreach(part; al.splitter(chunk, '\0')) { 4276 if(idx & 1) { // odd is value 4277 value = cast(string)(part.idup); 4278 headers[key] = value; // commit 4279 } else 4280 key = cast(string)(part.idup); 4281 idx++; 4282 } 4283 4284 enforce(chunk[size] == ','); // the terminator 4285 4286 range.consume(size + 1); 4287 // reading data 4288 // this will be done by Cgi 4289 4290 const(ubyte)[] getScgiChunk() { 4291 // we are already primed 4292 auto data = range.front(); 4293 if(data.length == 0 && !range.sourceClosed) { 4294 range.popFront(0); 4295 data = range.front(); 4296 } else if (range.sourceClosed) 4297 range.source.close(); 4298 4299 return data; 4300 } 4301 4302 void writeScgi(const(ubyte)[] data) { 4303 sendAll(connection, data); 4304 } 4305 4306 void flushScgi() { 4307 // I don't *think* I have to do anything.... 4308 } 4309 4310 Cgi cgi; 4311 try { 4312 cgi = new CustomCgi(maxContentLength, headers, &getScgiChunk, &writeScgi, &flushScgi); 4313 cgi._outputFileHandle = connection.handle; 4314 } catch(Throwable t) { 4315 sendAll(connection, plainHttpError(true, "400 Bad Request", t)); 4316 connection.close(); 4317 return; // this connection is dead 4318 } 4319 assert(cgi !is null); 4320 scope(exit) cgi.dispose(); 4321 try { 4322 fun(cgi); 4323 cgi.close(); 4324 connection.close(); 4325 } catch(Throwable t) { 4326 // no std err 4327 if(!handleException(cgi, t)) { 4328 connection.close(); 4329 return; 4330 } else { 4331 connection.close(); 4332 return; 4333 } 4334 } 4335 } 4336 4337 string printDate(DateTime date) { 4338 char[29] buffer = void; 4339 printDateToBuffer(date, buffer[]); 4340 return buffer.idup; 4341 } 4342 4343 int printDateToBuffer(DateTime date, char[] buffer) @nogc { 4344 assert(buffer.length >= 29); 4345 // 29 static length ? 4346 4347 static immutable daysOfWeek = [ 4348 "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" 4349 ]; 4350 4351 static immutable months = [ 4352 null, "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" 4353 ]; 4354 4355 buffer[0 .. 3] = daysOfWeek[date.dayOfWeek]; 4356 buffer[3 .. 5] = ", "; 4357 buffer[5] = date.day / 10 + '0'; 4358 buffer[6] = date.day % 10 + '0'; 4359 buffer[7] = ' '; 4360 buffer[8 .. 11] = months[date.month]; 4361 buffer[11] = ' '; 4362 auto y = date.year; 4363 buffer[12] = cast(char) (y / 1000 + '0'); y %= 1000; 4364 buffer[13] = cast(char) (y / 100 + '0'); y %= 100; 4365 buffer[14] = cast(char) (y / 10 + '0'); y %= 10; 4366 buffer[15] = cast(char) (y + '0'); 4367 buffer[16] = ' '; 4368 buffer[17] = date.hour / 10 + '0'; 4369 buffer[18] = date.hour % 10 + '0'; 4370 buffer[19] = ':'; 4371 buffer[20] = date.minute / 10 + '0'; 4372 buffer[21] = date.minute % 10 + '0'; 4373 buffer[22] = ':'; 4374 buffer[23] = date.second / 10 + '0'; 4375 buffer[24] = date.second % 10 + '0'; 4376 buffer[25 .. $] = " GMT"; 4377 4378 return 29; 4379 } 4380 4381 4382 // Referencing this gigantic typeid seems to remind the compiler 4383 // to actually put the symbol in the object file. I guess the immutable 4384 // assoc array array isn't actually included in druntime 4385 void hackAroundLinkerError() { 4386 stdout.rawWrite(typeid(const(immutable(char)[][])[immutable(char)[]]).toString()); 4387 stdout.rawWrite(typeid(immutable(char)[][][immutable(char)[]]).toString()); 4388 stdout.rawWrite(typeid(Cgi.UploadedFile[immutable(char)[]]).toString()); 4389 stdout.rawWrite(typeid(Cgi.UploadedFile[][immutable(char)[]]).toString()); 4390 stdout.rawWrite(typeid(immutable(Cgi.UploadedFile)[immutable(char)[]]).toString()); 4391 stdout.rawWrite(typeid(immutable(Cgi.UploadedFile[])[immutable(char)[]]).toString()); 4392 stdout.rawWrite(typeid(immutable(char[])[immutable(char)[]]).toString()); 4393 // this is getting kinda ridiculous btw. Moving assoc arrays 4394 // to the library is the pain that keeps on coming. 4395 4396 // eh this broke the build on the work server 4397 // stdout.rawWrite(typeid(immutable(char)[][immutable(string[])])); 4398 stdout.rawWrite(typeid(immutable(string[])[immutable(char)[]]).toString()); 4399 } 4400 4401 4402 4403 4404 4405 version(fastcgi) { 4406 pragma(lib, "fcgi"); 4407 4408 static if(size_t.sizeof == 8) // 64 bit 4409 alias long c_int; 4410 else 4411 alias int c_int; 4412 4413 extern(C) { 4414 struct FCGX_Stream { 4415 ubyte* rdNext; 4416 ubyte* wrNext; 4417 ubyte* stop; 4418 ubyte* stopUnget; 4419 c_int isReader; 4420 c_int isClosed; 4421 c_int wasFCloseCalled; 4422 c_int FCGI_errno; 4423 void* function(FCGX_Stream* stream) fillBuffProc; 4424 void* function(FCGX_Stream* stream, c_int doClose) emptyBuffProc; 4425 void* data; 4426 } 4427 4428 // note: this is meant to be opaque, so don't access it directly 4429 struct FCGX_Request { 4430 int requestId; 4431 int role; 4432 FCGX_Stream* inStream; 4433 FCGX_Stream* outStream; 4434 FCGX_Stream* errStream; 4435 char** envp; 4436 void* paramsPtr; 4437 int ipcFd; 4438 int isBeginProcessed; 4439 int keepConnection; 4440 int appStatus; 4441 int nWriters; 4442 int flags; 4443 int listen_sock; 4444 } 4445 4446 int FCGX_InitRequest(FCGX_Request *request, int sock, int flags); 4447 void FCGX_Init(); 4448 4449 int FCGX_Accept_r(FCGX_Request *request); 4450 4451 4452 alias char** FCGX_ParamArray; 4453 4454 c_int FCGX_Accept(FCGX_Stream** stdin, FCGX_Stream** stdout, FCGX_Stream** stderr, FCGX_ParamArray* envp); 4455 c_int FCGX_GetChar(FCGX_Stream* stream); 4456 c_int FCGX_PutStr(const ubyte* str, c_int n, FCGX_Stream* stream); 4457 int FCGX_HasSeenEOF(FCGX_Stream* stream); 4458 c_int FCGX_FFlush(FCGX_Stream *stream); 4459 4460 int FCGX_OpenSocket(in char*, int); 4461 } 4462 } 4463 4464 4465 /* This might go int a separate module eventually. It is a network input helper class. */ 4466 4467 import std.socket; 4468 4469 version(cgi_use_fiber) { 4470 import core.thread; 4471 import core.sys.linux.epoll; 4472 4473 __gshared int epfd; 4474 } 4475 4476 4477 version(cgi_use_fiber) 4478 private enum WakeupEvent { 4479 Read = EPOLLIN, 4480 Write = EPOLLOUT 4481 } 4482 4483 version(cgi_use_fiber) 4484 private void registerEventWakeup(bool* registered, Socket source, WakeupEvent e) @nogc { 4485 4486 // static cast since I know what i have in here and don't want to pay for dynamic cast 4487 auto f = cast(CgiFiber) cast(void*) Fiber.getThis(); 4488 4489 f.setPostYield = () { 4490 if(*registered) { 4491 // rearm 4492 epoll_event evt; 4493 evt.events = e | EPOLLONESHOT; 4494 evt.data.ptr = cast(void*) f; 4495 if(epoll_ctl(epfd, EPOLL_CTL_MOD, source.handle, &evt) == -1) 4496 throw new Exception("epoll_ctl"); 4497 } else { 4498 // initial registration 4499 *registered = true ; 4500 int fd = source.handle; 4501 epoll_event evt; 4502 evt.events = e | EPOLLONESHOT; 4503 evt.data.ptr = cast(void*) f; 4504 if(epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &evt) == -1) 4505 throw new Exception("epoll_ctl"); 4506 } 4507 }; 4508 4509 Fiber.yield(); 4510 4511 f.setPostYield(null); 4512 } 4513 4514 version(cgi_use_fiber) 4515 void unregisterSource(Socket s) { 4516 epoll_event evt; 4517 epoll_ctl(epfd, EPOLL_CTL_DEL, s.handle(), &evt); 4518 } 4519 4520 // it is a class primarily for reference semantics 4521 // I might change this interface 4522 /// This is NOT ACTUALLY an input range! It is too different. Historical mistake kinda. 4523 class BufferedInputRange { 4524 version(Posix) 4525 this(int source, ubyte[] buffer = null) { 4526 this(new Socket(cast(socket_t) source, AddressFamily.INET), buffer); 4527 } 4528 4529 this(Socket source, ubyte[] buffer = null, bool allowGrowth = true) { 4530 // if they connect but never send stuff to us, we don't want it wasting the process 4531 // so setting a time out 4532 version(cgi_use_fiber) 4533 source.blocking = false; 4534 else 4535 source.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(3)); 4536 4537 this.source = source; 4538 if(buffer is null) { 4539 underlyingBuffer = new ubyte[4096]; 4540 this.allowGrowth = true; 4541 } else { 4542 underlyingBuffer = buffer; 4543 this.allowGrowth = allowGrowth; 4544 } 4545 4546 assert(underlyingBuffer.length); 4547 4548 // we assume view.ptr is always inside underlyingBuffer 4549 view = underlyingBuffer[0 .. 0]; 4550 4551 popFront(); // prime 4552 } 4553 4554 version(cgi_use_fiber) { 4555 bool registered; 4556 } 4557 4558 void dispose() { 4559 version(cgi_use_fiber) { 4560 if(registered) 4561 unregisterSource(source); 4562 } 4563 } 4564 4565 /** 4566 A slight difference from regular ranges is you can give it the maximum 4567 number of bytes to consume. 4568 4569 IMPORTANT NOTE: the default is to consume nothing, so if you don't call 4570 consume() yourself and use a regular foreach, it will infinitely loop! 4571 4572 The default is to do what a normal range does, and consume the whole buffer 4573 and wait for additional input. 4574 4575 You can also specify 0, to append to the buffer, or any other number 4576 to remove the front n bytes and wait for more. 4577 */ 4578 void popFront(size_t maxBytesToConsume = 0 /*size_t.max*/, size_t minBytesToSettleFor = 0, bool skipConsume = false) { 4579 if(sourceClosed) 4580 throw new ConnectionClosedException("can't get any more data from a closed source"); 4581 if(!skipConsume) 4582 consume(maxBytesToConsume); 4583 4584 // we might have to grow the buffer 4585 if(minBytesToSettleFor > underlyingBuffer.length || view.length == underlyingBuffer.length) { 4586 if(allowGrowth) { 4587 //import std.stdio; writeln("growth"); 4588 auto viewStart = view.ptr - underlyingBuffer.ptr; 4589 size_t growth = 4096; 4590 // make sure we have enough for what we're being asked for 4591 if(minBytesToSettleFor > 0 && minBytesToSettleFor - underlyingBuffer.length > growth) 4592 growth = minBytesToSettleFor - underlyingBuffer.length; 4593 //import std.stdio; writeln(underlyingBuffer.length, " ", viewStart, " ", view.length, " ", growth, " ", minBytesToSettleFor, " ", minBytesToSettleFor - underlyingBuffer.length); 4594 underlyingBuffer.length += growth; 4595 view = underlyingBuffer[viewStart .. view.length]; 4596 } else 4597 throw new Exception("No room left in the buffer"); 4598 } 4599 4600 do { 4601 auto freeSpace = underlyingBuffer[view.ptr - underlyingBuffer.ptr + view.length .. $]; 4602 try_again: 4603 auto ret = source.receive(freeSpace); 4604 if(ret == Socket.ERROR) { 4605 if(wouldHaveBlocked()) { 4606 version(cgi_use_fiber) { 4607 registerEventWakeup(®istered, source, WakeupEvent.Read); 4608 goto try_again; 4609 } else { 4610 // gonna treat a timeout here as a close 4611 sourceClosed = true; 4612 return; 4613 } 4614 } 4615 version(Posix) { 4616 import core.stdc.errno; 4617 if(errno == EINTR || errno == EAGAIN) { 4618 goto try_again; 4619 } 4620 if(errno == ECONNRESET) { 4621 sourceClosed = true; 4622 return; 4623 } 4624 } 4625 throw new Exception(lastSocketError); // FIXME 4626 } 4627 if(ret == 0) { 4628 sourceClosed = true; 4629 return; 4630 } 4631 4632 //import std.stdio; writeln(view.ptr); writeln(underlyingBuffer.ptr); writeln(view.length, " ", ret, " = ", view.length + ret); 4633 view = underlyingBuffer[view.ptr - underlyingBuffer.ptr .. view.length + ret]; 4634 //import std.stdio; writeln(cast(string) view); 4635 } while(view.length < minBytesToSettleFor); 4636 } 4637 4638 /// Removes n bytes from the front of the buffer, and returns the new buffer slice. 4639 /// You might want to idup the data you are consuming if you store it, since it may 4640 /// be overwritten on the new popFront. 4641 /// 4642 /// You do not need to call this if you always want to wait for more data when you 4643 /// consume some. 4644 ubyte[] consume(size_t bytes) { 4645 //import std.stdio; writeln("consuime ", bytes, "/", view.length); 4646 view = view[bytes > $ ? $ : bytes .. $]; 4647 if(view.length == 0) { 4648 view = underlyingBuffer[0 .. 0]; // go ahead and reuse the beginning 4649 /* 4650 writeln("HERE"); 4651 popFront(0, 0, true); // try to load more if we can, checks if the source is closed 4652 writeln(cast(string)front); 4653 writeln("DONE"); 4654 */ 4655 } 4656 return front; 4657 } 4658 4659 bool empty() { 4660 return sourceClosed && view.length == 0; 4661 } 4662 4663 ubyte[] front() { 4664 return view; 4665 } 4666 4667 invariant() { 4668 assert(view.ptr >= underlyingBuffer.ptr); 4669 // it should never be equal, since if that happens view ought to be empty, and thus reusing the buffer 4670 assert(view.ptr < underlyingBuffer.ptr + underlyingBuffer.length); 4671 } 4672 4673 ubyte[] underlyingBuffer; 4674 bool allowGrowth; 4675 ubyte[] view; 4676 Socket source; 4677 bool sourceClosed; 4678 } 4679 4680 private class FakeSocketForStdin : Socket { 4681 import std.stdio; 4682 4683 this() { 4684 4685 } 4686 4687 private bool closed; 4688 4689 override ptrdiff_t receive(void[] buffer, std.socket.SocketFlags) @trusted { 4690 if(closed) 4691 throw new Exception("Closed"); 4692 return stdin.rawRead(buffer).length; 4693 } 4694 4695 override ptrdiff_t send(const void[] buffer, std.socket.SocketFlags) @trusted { 4696 if(closed) 4697 throw new Exception("Closed"); 4698 stdout.rawWrite(buffer); 4699 return buffer.length; 4700 } 4701 4702 override void close() @trusted { 4703 (cast(void delegate() @nogc nothrow) &realClose)(); 4704 } 4705 4706 override void shutdown(SocketShutdown s) { 4707 // FIXME 4708 } 4709 4710 override void setOption(SocketOptionLevel, SocketOption, void[]) {} 4711 override void setOption(SocketOptionLevel, SocketOption, Duration) {} 4712 4713 override @property @trusted Address remoteAddress() { return null; } 4714 override @property @trusted Address localAddress() { return null; } 4715 4716 void realClose() { 4717 closed = true; 4718 try { 4719 stdin.close(); 4720 stdout.close(); 4721 } catch(Exception e) { 4722 4723 } 4724 } 4725 } 4726 4727 import core.sync.semaphore; 4728 import core.atomic; 4729 4730 /** 4731 To use this thing: 4732 4733 void handler(Socket s) { do something... } 4734 auto manager = new ListeningConnectionManager("127.0.0.1", 80, &handler); 4735 manager.listen(); 4736 4737 I suggest you use BufferedInputRange(connection) to handle the input. As a packet 4738 comes in, you will get control. You can just continue; though to fetch more. 4739 4740 4741 FIXME: should I offer an event based async thing like netman did too? Yeah, probably. 4742 */ 4743 class ListeningConnectionManager { 4744 Semaphore semaphore; 4745 Socket[256] queue; 4746 shared(ubyte) nextIndexFront; 4747 ubyte nextIndexBack; 4748 shared(int) queueLength; 4749 4750 void listen() { 4751 running = true; 4752 shared(int) loopBroken; 4753 4754 version(Posix) { 4755 import core.sys.posix.signal; 4756 signal(SIGPIPE, SIG_IGN); 4757 } 4758 4759 version(cgi_no_threads) { 4760 // NEVER USE THIS 4761 // it exists only for debugging and other special occasions 4762 4763 // the thread mode is faster and less likely to stall the whole 4764 // thing when a request is slow 4765 while(!loopBroken && running) { 4766 auto sn = listener.accept(); 4767 cloexec(sn); 4768 try { 4769 handler(sn); 4770 } catch(Exception e) { 4771 // if a connection goes wrong, we want to just say no, but try to carry on unless it is an Error of some sort (in which case, we'll die. You might want an external helper program to revive the server when it dies) 4772 sn.close(); 4773 } 4774 } 4775 } else { 4776 import std.parallelism; 4777 4778 version(cgi_use_fork) { 4779 //asm { int 3; } 4780 fork(); 4781 } 4782 4783 version(cgi_use_fiber) { 4784 import core.sys.linux.epoll; 4785 epfd = epoll_create1(EPOLL_CLOEXEC); 4786 if(epfd == -1) 4787 throw new Exception("epoll_create1 " ~ to!string(errno)); 4788 scope(exit) { 4789 import core.sys.posix.unistd; 4790 close(epfd); 4791 } 4792 4793 epoll_event ev; 4794 ev.events = EPOLLIN | EPOLLEXCLUSIVE; // EPOLLEXCLUSIVE is only available on kernels since like 2017 but that's prolly good enough. 4795 ev.data.fd = listener.handle; 4796 if(epoll_ctl(epfd, EPOLL_CTL_ADD, listener.handle, &ev) == -1) 4797 throw new Exception("epoll_ctl " ~ to!string(errno)); 4798 4799 WorkerThread[] threads = new WorkerThread[](totalCPUs * 1 + 1); 4800 foreach(i, ref thread; threads) { 4801 thread = new WorkerThread(this, handler, cast(int) i); 4802 thread.start(); 4803 } 4804 4805 bool fiber_crash_check() { 4806 bool hasAnyRunning; 4807 foreach(thread; threads) { 4808 if(!thread.isRunning) { 4809 thread.join(); 4810 } else hasAnyRunning = true; 4811 } 4812 4813 return (!hasAnyRunning); 4814 } 4815 4816 4817 while(running) { 4818 Thread.sleep(1.seconds); 4819 if(fiber_crash_check()) 4820 break; 4821 } 4822 4823 } else { 4824 semaphore = new Semaphore(); 4825 4826 // I times 4 here because there's a good chance some will be blocked on i/o. 4827 ConnectionThread[] threads = new ConnectionThread[](totalCPUs * 4); 4828 foreach(i, ref thread; threads) { 4829 thread = new ConnectionThread(this, handler, cast(int) i); 4830 thread.start(); 4831 } 4832 } 4833 4834 while(!loopBroken && running) { 4835 Socket sn; 4836 4837 bool crash_check() { 4838 bool hasAnyRunning; 4839 foreach(thread; threads) { 4840 if(!thread.isRunning) { 4841 thread.join(); 4842 } else hasAnyRunning = true; 4843 } 4844 4845 return (!hasAnyRunning); 4846 } 4847 4848 4849 void accept_new_connection() { 4850 sn = listener.accept(); 4851 cloexec(sn); 4852 if(tcp) { 4853 // disable Nagle's algorithm to avoid a 40ms delay when we send/recv 4854 // on the socket because we do some buffering internally. I think this helps, 4855 // certainly does for small requests, and I think it does for larger ones too 4856 sn.setOption(SocketOptionLevel.TCP, SocketOption.TCP_NODELAY, 1); 4857 4858 sn.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(10)); 4859 } 4860 } 4861 4862 void existing_connection_new_data() { 4863 // wait until a slot opens up 4864 //int waited = 0; 4865 while(queueLength >= queue.length) { 4866 Thread.sleep(1.msecs); 4867 //waited ++; 4868 } 4869 //if(waited) {import std.stdio; writeln(waited);} 4870 synchronized(this) { 4871 queue[nextIndexBack] = sn; 4872 nextIndexBack++; 4873 atomicOp!"+="(queueLength, 1); 4874 } 4875 semaphore.notify(); 4876 } 4877 4878 4879 accept_new_connection(); 4880 existing_connection_new_data(); 4881 4882 if(crash_check()) 4883 break; 4884 } 4885 4886 // FIXME: i typically stop this with ctrl+c which never 4887 // actually gets here. i need to do a sigint handler. 4888 if(cleanup) 4889 cleanup(); 4890 } 4891 } 4892 4893 //version(linux) 4894 //int epoll_fd; 4895 4896 bool tcp; 4897 void delegate() cleanup; 4898 4899 private void function(Socket) fhandler; 4900 private void dg_handler(Socket s) { 4901 fhandler(s); 4902 } 4903 this(string host, ushort port, void function(Socket) handler) { 4904 fhandler = handler; 4905 this(host, port, &dg_handler); 4906 } 4907 4908 this(string host, ushort port, void delegate(Socket) handler) { 4909 this.handler = handler; 4910 4911 listener = startListening(host, port, tcp, cleanup, 128); 4912 4913 version(cgi_use_fiber) version(cgi_use_fork) 4914 listener.blocking = false; 4915 4916 // this is the UI control thread and thus gets more priority 4917 Thread.getThis.priority = Thread.PRIORITY_MAX; 4918 } 4919 4920 Socket listener; 4921 void delegate(Socket) handler; 4922 4923 bool running; 4924 void quit() { 4925 running = false; 4926 } 4927 } 4928 4929 Socket startListening(string host, ushort port, ref bool tcp, ref void delegate() cleanup, int backQueue) { 4930 Socket listener; 4931 if(host.startsWith("unix:")) { 4932 version(Posix) { 4933 listener = new Socket(AddressFamily.UNIX, SocketType.STREAM); 4934 cloexec(listener); 4935 string filename = host["unix:".length .. $].idup; 4936 listener.bind(new UnixAddress(filename)); 4937 cleanup = delegate() { 4938 listener.close(); 4939 import std.file; 4940 remove(filename); 4941 }; 4942 tcp = false; 4943 } else { 4944 throw new Exception("unix sockets not supported on this system"); 4945 } 4946 } else if(host.startsWith("abstract:")) { 4947 version(linux) { 4948 listener = new Socket(AddressFamily.UNIX, SocketType.STREAM); 4949 cloexec(listener); 4950 string filename = "\0" ~ host["abstract:".length .. $]; 4951 import std.stdio; stderr.writeln("Listening to abstract unix domain socket: ", host["abstract:".length .. $]); 4952 listener.bind(new UnixAddress(filename)); 4953 tcp = false; 4954 } else { 4955 throw new Exception("abstract unix sockets not supported on this system"); 4956 } 4957 } else { 4958 listener = new TcpSocket(); 4959 cloexec(listener); 4960 listener.setOption(SocketOptionLevel.SOCKET, SocketOption.REUSEADDR, true); 4961 listener.bind(host.length ? parseAddress(host, port) : new InternetAddress(port)); 4962 cleanup = delegate() { 4963 listener.close(); 4964 }; 4965 tcp = true; 4966 } 4967 4968 listener.listen(backQueue); 4969 4970 dropPrivs(); 4971 4972 return listener; 4973 } 4974 4975 // helper function to send a lot to a socket. Since this blocks for the buffer (possibly several times), you should probably call it in a separate thread or something. 4976 void sendAll(Socket s, const(void)[] data, string file = __FILE__, size_t line = __LINE__) { 4977 if(data.length == 0) return; 4978 ptrdiff_t amount; 4979 //import std.stdio; writeln("***",cast(string) data,"///"); 4980 do { 4981 amount = s.send(data); 4982 if(amount == Socket.ERROR) { 4983 version(cgi_use_fiber) { 4984 if(wouldHaveBlocked()) { 4985 bool registered = true; 4986 registerEventWakeup(®istered, s, WakeupEvent.Write); 4987 continue; 4988 } 4989 } 4990 throw new ConnectionException(s, lastSocketError, file, line); 4991 } 4992 assert(amount > 0); 4993 data = data[amount .. $]; 4994 } while(data.length); 4995 } 4996 4997 class ConnectionException : Exception { 4998 Socket socket; 4999 this(Socket s, string msg, string file = __FILE__, size_t line = __LINE__) { 5000 this.socket = s; 5001 super(msg, file, line); 5002 } 5003 } 5004 5005 alias void delegate(Socket) CMT; 5006 5007 import core.thread; 5008 /+ 5009 cgi.d now uses a hybrid of event i/o and threads at the top level. 5010 5011 Top level thread is responsible for accepting sockets and selecting on them. 5012 5013 It then indicates to a child that a request is pending, and any random worker 5014 thread that is free handles it. It goes into blocking mode and handles that 5015 http request to completion. 5016 5017 At that point, it goes back into the waiting queue. 5018 5019 5020 This concept is only implemented on Linux. On all other systems, it still 5021 uses the worker threads and semaphores (which is perfectly fine for a lot of 5022 things! Just having a great number of keep-alive connections will break that.) 5023 5024 5025 So the algorithm is: 5026 5027 select(accept, event, pending) 5028 if accept -> send socket to free thread, if any. if not, add socket to queue 5029 if event -> send the signaling thread a socket from the queue, if not, mark it free 5030 - event might block until it can be *written* to. it is a fifo sending socket fds! 5031 5032 A worker only does one http request at a time, then signals its availability back to the boss. 5033 5034 The socket the worker was just doing should be added to the one-off epoll read. If it is closed, 5035 great, we can get rid of it. Otherwise, it is considered `pending`. The *kernel* manages that; the 5036 actual FD will not be kept out here. 5037 5038 So: 5039 queue = sockets we know are ready to read now, but no worker thread is available 5040 idle list = worker threads not doing anything else. they signal back and forth 5041 5042 the workers all read off the event fd. This is the semaphore wait 5043 5044 the boss waits on accept or other sockets read events (one off! and level triggered). If anything happens wrt ready read, 5045 it puts it in the queue and writes to the event fd. 5046 5047 The child could put the socket back in the epoll thing itself. 5048 5049 The child needs to be able to gracefully handle being given a socket that just closed with no work. 5050 +/ 5051 class ConnectionThread : Thread { 5052 this(ListeningConnectionManager lcm, CMT dg, int myThreadNumber) { 5053 this.lcm = lcm; 5054 this.dg = dg; 5055 this.myThreadNumber = myThreadNumber; 5056 super(&run); 5057 } 5058 5059 void run() { 5060 while(true) { 5061 // so if there's a bunch of idle keep-alive connections, it can 5062 // consume all the worker threads... just sitting there. 5063 lcm.semaphore.wait(); 5064 Socket socket; 5065 synchronized(lcm) { 5066 auto idx = lcm.nextIndexFront; 5067 socket = lcm.queue[idx]; 5068 lcm.queue[idx] = null; 5069 atomicOp!"+="(lcm.nextIndexFront, 1); 5070 atomicOp!"-="(lcm.queueLength, 1); 5071 } 5072 try { 5073 //import std.stdio; writeln(myThreadNumber, " taking it"); 5074 dg(socket); 5075 /+ 5076 if(socket.isAlive) { 5077 // process it more later 5078 version(linux) { 5079 import core.sys.linux.epoll; 5080 epoll_event ev; 5081 ev.events = EPOLLIN | EPOLLONESHOT | EPOLLET; 5082 ev.data.fd = socket.handle; 5083 import std.stdio; writeln("adding"); 5084 if(epoll_ctl(lcm.epoll_fd, EPOLL_CTL_ADD, socket.handle, &ev) == -1) { 5085 if(errno == EEXIST) { 5086 ev.events = EPOLLIN | EPOLLONESHOT | EPOLLET; 5087 ev.data.fd = socket.handle; 5088 if(epoll_ctl(lcm.epoll_fd, EPOLL_CTL_MOD, socket.handle, &ev) == -1) 5089 throw new Exception("epoll_ctl " ~ to!string(errno)); 5090 } else 5091 throw new Exception("epoll_ctl " ~ to!string(errno)); 5092 } 5093 //import std.stdio; writeln("keep alive"); 5094 // writing to this private member is to prevent the GC from closing my precious socket when I'm trying to use it later 5095 __traits(getMember, socket, "sock") = cast(socket_t) -1; 5096 } else { 5097 continue; // hope it times out in a reasonable amount of time... 5098 } 5099 } 5100 +/ 5101 } catch(ConnectionClosedException e) { 5102 // can just ignore this, it is fairly normal 5103 socket.close(); 5104 } catch(Throwable e) { 5105 import std.stdio; stderr.rawWrite(e.toString); stderr.rawWrite("\n"); 5106 socket.close(); 5107 } 5108 } 5109 } 5110 5111 ListeningConnectionManager lcm; 5112 CMT dg; 5113 int myThreadNumber; 5114 } 5115 5116 version(cgi_use_fiber) 5117 class WorkerThread : Thread { 5118 this(ListeningConnectionManager lcm, CMT dg, int myThreadNumber) { 5119 this.lcm = lcm; 5120 this.dg = dg; 5121 this.myThreadNumber = myThreadNumber; 5122 super(&run); 5123 } 5124 5125 void run() { 5126 while(lcm.running) { 5127 Socket sn; 5128 5129 epoll_event[64] events; 5130 auto nfds = epoll_wait(epfd, events.ptr, events.length, -1); 5131 if(nfds == -1) { 5132 if(errno == EINTR) 5133 continue; 5134 throw new Exception("epoll_wait " ~ to!string(errno)); 5135 } 5136 5137 foreach(idx; 0 .. nfds) { 5138 auto flags = events[idx].events; 5139 5140 if(cast(size_t) events[idx].data.ptr == cast(size_t) lcm.listener.handle) { 5141 // this try/catch is because it is set to non-blocking mode 5142 // and Phobos' stupid api throws an exception instead of returning 5143 // if it would block. Why would it block? because a forked process 5144 // might have beat us to it, but the wakeup event thundered our herds. 5145 version(cgi_use_fork) { 5146 try 5147 sn = lcm.listener.accept(); 5148 catch(SocketAcceptException e) { continue; } 5149 } else { 5150 sn = lcm.listener.accept(); 5151 } 5152 5153 cloexec(sn); 5154 if(lcm.tcp) { 5155 // disable Nagle's algorithm to avoid a 40ms delay when we send/recv 5156 // on the socket because we do some buffering internally. I think this helps, 5157 // certainly does for small requests, and I think it does for larger ones too 5158 sn.setOption(SocketOptionLevel.TCP, SocketOption.TCP_NODELAY, 1); 5159 5160 sn.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"seconds"(10)); 5161 } 5162 5163 dg(sn); 5164 } else { 5165 auto fiber = cast(CgiFiber) events[idx].data.ptr; 5166 fiber.proceed(); 5167 } 5168 } 5169 } 5170 } 5171 5172 ListeningConnectionManager lcm; 5173 CMT dg; 5174 int myThreadNumber; 5175 } 5176 5177 5178 /* Done with network helper */ 5179 5180 /* Helpers for doing temporary files. Used both here and in web.d */ 5181 5182 version(Windows) { 5183 import core.sys.windows.windows; 5184 extern(Windows) DWORD GetTempPathW(DWORD, LPWSTR); 5185 alias GetTempPathW GetTempPath; 5186 } 5187 5188 version(Posix) { 5189 static import linux = core.sys.posix.unistd; 5190 } 5191 5192 string getTempDirectory() { 5193 string path; 5194 version(Windows) { 5195 wchar[1024] buffer; 5196 auto len = GetTempPath(1024, buffer.ptr); 5197 if(len == 0) 5198 throw new Exception("couldn't find a temporary path"); 5199 5200 auto b = buffer[0 .. len]; 5201 5202 path = to!string(b); 5203 } else 5204 path = "/tmp/"; 5205 5206 return path; 5207 } 5208 5209 5210 // I like std.date. These functions help keep my old code and data working with phobos changing. 5211 5212 long sysTimeToDTime(in SysTime sysTime) { 5213 return convert!("hnsecs", "msecs")(sysTime.stdTime - 621355968000000000L); 5214 } 5215 5216 long dateTimeToDTime(in DateTime dt) { 5217 return sysTimeToDTime(cast(SysTime) dt); 5218 } 5219 5220 long getUtcTime() { // renamed primarily to avoid conflict with std.date itself 5221 return sysTimeToDTime(Clock.currTime(UTC())); 5222 } 5223 5224 // NOTE: new SimpleTimeZone(minutes); can perhaps work with the getTimezoneOffset() JS trick 5225 SysTime dTimeToSysTime(long dTime, immutable TimeZone tz = null) { 5226 immutable hnsecs = convert!("msecs", "hnsecs")(dTime) + 621355968000000000L; 5227 return SysTime(hnsecs, tz); 5228 } 5229 5230 5231 5232 // this is a helper to read HTTP transfer-encoding: chunked responses 5233 immutable(ubyte[]) dechunk(BufferedInputRange ir) { 5234 immutable(ubyte)[] ret; 5235 5236 another_chunk: 5237 // If here, we are at the beginning of a chunk. 5238 auto a = ir.front(); 5239 int chunkSize; 5240 int loc = locationOf(a, "\r\n"); 5241 while(loc == -1) { 5242 ir.popFront(); 5243 a = ir.front(); 5244 loc = locationOf(a, "\r\n"); 5245 } 5246 5247 string hex; 5248 hex = ""; 5249 for(int i = 0; i < loc; i++) { 5250 char c = a[i]; 5251 if(c >= 'A' && c <= 'Z') 5252 c += 0x20; 5253 if((c >= '0' && c <= '9') || (c >= 'a' && c <= 'z')) { 5254 hex ~= c; 5255 } else { 5256 break; 5257 } 5258 } 5259 5260 assert(hex.length); 5261 5262 int power = 1; 5263 int size = 0; 5264 foreach(cc1; retro(hex)) { 5265 dchar cc = cc1; 5266 if(cc >= 'a' && cc <= 'z') 5267 cc -= 0x20; 5268 int val = 0; 5269 if(cc >= '0' && cc <= '9') 5270 val = cc - '0'; 5271 else 5272 val = cc - 'A' + 10; 5273 5274 size += power * val; 5275 power *= 16; 5276 } 5277 5278 chunkSize = size; 5279 assert(size >= 0); 5280 5281 if(loc + 2 > a.length) { 5282 ir.popFront(0, a.length + loc + 2); 5283 a = ir.front(); 5284 } 5285 5286 a = ir.consume(loc + 2); 5287 5288 if(chunkSize == 0) { // we're done with the response 5289 // if we got here, will change must be true.... 5290 more_footers: 5291 loc = locationOf(a, "\r\n"); 5292 if(loc == -1) { 5293 ir.popFront(); 5294 a = ir.front; 5295 goto more_footers; 5296 } else { 5297 assert(loc == 0); 5298 ir.consume(loc + 2); 5299 goto finish; 5300 } 5301 } else { 5302 // if we got here, will change must be true.... 5303 if(a.length < chunkSize + 2) { 5304 ir.popFront(0, chunkSize + 2); 5305 a = ir.front(); 5306 } 5307 5308 ret ~= (a[0..chunkSize]); 5309 5310 if(!(a.length > chunkSize + 2)) { 5311 ir.popFront(0, chunkSize + 2); 5312 a = ir.front(); 5313 } 5314 assert(a[chunkSize] == 13); 5315 assert(a[chunkSize+1] == 10); 5316 a = ir.consume(chunkSize + 2); 5317 chunkSize = 0; 5318 goto another_chunk; 5319 } 5320 5321 finish: 5322 return ret; 5323 } 5324 5325 // I want to be able to get data from multiple sources the same way... 5326 interface ByChunkRange { 5327 bool empty(); 5328 void popFront(); 5329 const(ubyte)[] front(); 5330 } 5331 5332 ByChunkRange byChunk(const(ubyte)[] data) { 5333 return new class ByChunkRange { 5334 override bool empty() { 5335 return !data.length; 5336 } 5337 5338 override void popFront() { 5339 if(data.length > 4096) 5340 data = data[4096 .. $]; 5341 else 5342 data = null; 5343 } 5344 5345 override const(ubyte)[] front() { 5346 return data[0 .. $ > 4096 ? 4096 : $]; 5347 } 5348 }; 5349 } 5350 5351 ByChunkRange byChunk(BufferedInputRange ir, size_t atMost) { 5352 const(ubyte)[] f; 5353 5354 f = ir.front; 5355 if(f.length > atMost) 5356 f = f[0 .. atMost]; 5357 5358 return new class ByChunkRange { 5359 override bool empty() { 5360 return atMost == 0; 5361 } 5362 5363 override const(ubyte)[] front() { 5364 return f; 5365 } 5366 5367 override void popFront() { 5368 ir.consume(f.length); 5369 atMost -= f.length; 5370 auto a = ir.front(); 5371 5372 if(a.length <= atMost) { 5373 f = a; 5374 atMost -= a.length; 5375 a = ir.consume(a.length); 5376 if(atMost != 0) 5377 ir.popFront(); 5378 if(f.length == 0) { 5379 f = ir.front(); 5380 } 5381 } else { 5382 // we actually have *more* here than we need.... 5383 f = a[0..atMost]; 5384 atMost = 0; 5385 ir.consume(atMost); 5386 } 5387 } 5388 }; 5389 } 5390 5391 version(cgi_with_websocket) { 5392 // http://tools.ietf.org/html/rfc6455 5393 5394 /** 5395 WEBSOCKET SUPPORT: 5396 5397 Full example: 5398 --- 5399 import arsd.cgi; 5400 5401 void websocketEcho(Cgi cgi) { 5402 if(cgi.websocketRequested()) { 5403 if(cgi.origin != "http://arsdnet.net") 5404 throw new Exception("bad origin"); 5405 auto websocket = cgi.acceptWebsocket(); 5406 5407 websocket.send("hello"); 5408 websocket.send(" world!"); 5409 5410 auto msg = websocket.recv(); 5411 while(msg.opcode != WebSocketOpcode.close) { 5412 if(msg.opcode == WebSocketOpcode.text) { 5413 websocket.send(msg.textData); 5414 } else if(msg.opcode == WebSocketOpcode.binary) { 5415 websocket.send(msg.data); 5416 } 5417 5418 msg = websocket.recv(); 5419 } 5420 5421 websocket.close(); 5422 } else assert(0, "i want a web socket!"); 5423 } 5424 5425 mixin GenericMain!websocketEcho; 5426 --- 5427 */ 5428 5429 class WebSocket { 5430 Cgi cgi; 5431 5432 private this(Cgi cgi) { 5433 this.cgi = cgi; 5434 5435 Socket socket = cgi.idlol.source; 5436 socket.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVTIMEO, dur!"minutes"(5)); 5437 } 5438 5439 // returns true if data available, false if it timed out 5440 bool recvAvailable(Duration timeout = dur!"msecs"(0)) { 5441 if(!waitForNextMessageWouldBlock()) 5442 return true; 5443 if(isDataPending(timeout)) 5444 return true; // this is kinda a lie. 5445 5446 return false; 5447 } 5448 5449 public bool lowLevelReceive() { 5450 auto bfr = cgi.idlol; 5451 top: 5452 auto got = bfr.front; 5453 if(got.length) { 5454 if(receiveBuffer.length < receiveBufferUsedLength + got.length) 5455 receiveBuffer.length += receiveBufferUsedLength + got.length; 5456 5457 receiveBuffer[receiveBufferUsedLength .. receiveBufferUsedLength + got.length] = got[]; 5458 receiveBufferUsedLength += got.length; 5459 bfr.consume(got.length); 5460 5461 return true; 5462 } 5463 5464 if(bfr.sourceClosed) 5465 return false; 5466 5467 bfr.popFront(0); 5468 if(bfr.sourceClosed) 5469 return false; 5470 goto top; 5471 } 5472 5473 5474 bool isDataPending(Duration timeout = 0.seconds) { 5475 Socket socket = cgi.idlol.source; 5476 5477 auto check = new SocketSet(); 5478 check.add(socket); 5479 5480 auto got = Socket.select(check, null, null, timeout); 5481 if(got > 0) 5482 return true; 5483 return false; 5484 } 5485 5486 // note: this blocks 5487 WebSocketFrame recv() { 5488 return waitForNextMessage(); 5489 } 5490 5491 5492 5493 5494 private void llclose() { 5495 cgi.close(); 5496 } 5497 5498 private void llsend(ubyte[] data) { 5499 cgi.write(data); 5500 cgi.flush(); 5501 } 5502 5503 void unregisterActiveSocket(WebSocket) {} 5504 5505 /* copy/paste section { */ 5506 5507 private int readyState_; 5508 private ubyte[] receiveBuffer; 5509 private size_t receiveBufferUsedLength; 5510 5511 private Config config; 5512 5513 enum CONNECTING = 0; /// Socket has been created. The connection is not yet open. 5514 enum OPEN = 1; /// The connection is open and ready to communicate. 5515 enum CLOSING = 2; /// The connection is in the process of closing. 5516 enum CLOSED = 3; /// The connection is closed or couldn't be opened. 5517 5518 /++ 5519 5520 +/ 5521 /// Group: foundational 5522 static struct Config { 5523 /++ 5524 These control the size of the receive buffer. 5525 5526 It starts at the initial size, will temporarily 5527 balloon up to the maximum size, and will reuse 5528 a buffer up to the likely size. 5529 5530 Anything larger than the maximum size will cause 5531 the connection to be aborted and an exception thrown. 5532 This is to protect you against a peer trying to 5533 exhaust your memory, while keeping the user-level 5534 processing simple. 5535 +/ 5536 size_t initialReceiveBufferSize = 4096; 5537 size_t likelyReceiveBufferSize = 4096; /// ditto 5538 size_t maximumReceiveBufferSize = 10 * 1024 * 1024; /// ditto 5539 5540 /++ 5541 Maximum combined size of a message. 5542 +/ 5543 size_t maximumMessageSize = 10 * 1024 * 1024; 5544 5545 string[string] cookies; /// Cookies to send with the initial request. cookies[name] = value; 5546 string origin; /// Origin URL to send with the handshake, if desired. 5547 string protocol; /// the protocol header, if desired. 5548 5549 int pingFrequency = 5000; /// Amount of time (in msecs) of idleness after which to send an automatic ping 5550 } 5551 5552 /++ 5553 Returns one of [CONNECTING], [OPEN], [CLOSING], or [CLOSED]. 5554 +/ 5555 int readyState() { 5556 return readyState_; 5557 } 5558 5559 /++ 5560 Closes the connection, sending a graceful teardown message to the other side. 5561 +/ 5562 /// Group: foundational 5563 void close(int code = 0, string reason = null) 5564 //in (reason.length < 123) 5565 in { assert(reason.length < 123); } do 5566 { 5567 if(readyState_ != OPEN) 5568 return; // it cool, we done 5569 WebSocketFrame wss; 5570 wss.fin = true; 5571 wss.opcode = WebSocketOpcode.close; 5572 wss.data = cast(ubyte[]) reason; 5573 wss.send(&llsend); 5574 5575 readyState_ = CLOSING; 5576 5577 llclose(); 5578 } 5579 5580 /++ 5581 Sends a ping message to the server. This is done automatically by the library if you set a non-zero [Config.pingFrequency], but you can also send extra pings explicitly as well with this function. 5582 +/ 5583 /// Group: foundational 5584 void ping() { 5585 WebSocketFrame wss; 5586 wss.fin = true; 5587 wss.opcode = WebSocketOpcode.ping; 5588 wss.send(&llsend); 5589 } 5590 5591 // automatically handled.... 5592 void pong() { 5593 WebSocketFrame wss; 5594 wss.fin = true; 5595 wss.opcode = WebSocketOpcode.pong; 5596 wss.send(&llsend); 5597 } 5598 5599 /++ 5600 Sends a text message through the websocket. 5601 +/ 5602 /// Group: foundational 5603 void send(in char[] textData) { 5604 WebSocketFrame wss; 5605 wss.fin = true; 5606 wss.opcode = WebSocketOpcode.text; 5607 wss.data = cast(ubyte[]) textData; 5608 wss.send(&llsend); 5609 } 5610 5611 /++ 5612 Sends a binary message through the websocket. 5613 +/ 5614 /// Group: foundational 5615 void send(in ubyte[] binaryData) { 5616 WebSocketFrame wss; 5617 wss.fin = true; 5618 wss.opcode = WebSocketOpcode.binary; 5619 wss.data = cast(ubyte[]) binaryData; 5620 wss.send(&llsend); 5621 } 5622 5623 /++ 5624 Waits for and returns the next complete message on the socket. 5625 5626 Note that the onmessage function is still called, right before 5627 this returns. 5628 +/ 5629 /// Group: blocking_api 5630 public WebSocketFrame waitForNextMessage() { 5631 do { 5632 auto m = processOnce(); 5633 if(m.populated) 5634 return m; 5635 } while(lowLevelReceive()); 5636 5637 throw new ConnectionClosedException("Websocket receive timed out"); 5638 //return WebSocketFrame.init; // FIXME? maybe. 5639 } 5640 5641 /++ 5642 Tells if [waitForNextMessage] would block. 5643 +/ 5644 /// Group: blocking_api 5645 public bool waitForNextMessageWouldBlock() { 5646 checkAgain: 5647 if(isMessageBuffered()) 5648 return false; 5649 if(!isDataPending()) 5650 return true; 5651 while(isDataPending()) 5652 lowLevelReceive(); 5653 goto checkAgain; 5654 } 5655 5656 /++ 5657 Is there a message in the buffer already? 5658 If `true`, [waitForNextMessage] is guaranteed to return immediately. 5659 If `false`, check [isDataPending] as the next step. 5660 +/ 5661 /// Group: blocking_api 5662 public bool isMessageBuffered() { 5663 ubyte[] d = receiveBuffer[0 .. receiveBufferUsedLength]; 5664 auto s = d; 5665 if(d.length) { 5666 auto orig = d; 5667 auto m = WebSocketFrame.read(d); 5668 // that's how it indicates that it needs more data 5669 if(d !is orig) 5670 return true; 5671 } 5672 5673 return false; 5674 } 5675 5676 private ubyte continuingType; 5677 private ubyte[] continuingData; 5678 //private size_t continuingDataLength; 5679 5680 private WebSocketFrame processOnce() { 5681 ubyte[] d = receiveBuffer[0 .. receiveBufferUsedLength]; 5682 auto s = d; 5683 // FIXME: handle continuation frames more efficiently. it should really just reuse the receive buffer. 5684 WebSocketFrame m; 5685 if(d.length) { 5686 auto orig = d; 5687 m = WebSocketFrame.read(d); 5688 // that's how it indicates that it needs more data 5689 if(d is orig) 5690 return WebSocketFrame.init; 5691 m.unmaskInPlace(); 5692 switch(m.opcode) { 5693 case WebSocketOpcode.continuation: 5694 if(continuingData.length + m.data.length > config.maximumMessageSize) 5695 throw new Exception("message size exceeded"); 5696 5697 continuingData ~= m.data; 5698 if(m.fin) { 5699 if(ontextmessage) 5700 ontextmessage(cast(char[]) continuingData); 5701 if(onbinarymessage) 5702 onbinarymessage(continuingData); 5703 5704 continuingData = null; 5705 } 5706 break; 5707 case WebSocketOpcode.text: 5708 if(m.fin) { 5709 if(ontextmessage) 5710 ontextmessage(m.textData); 5711 } else { 5712 continuingType = m.opcode; 5713 //continuingDataLength = 0; 5714 continuingData = null; 5715 continuingData ~= m.data; 5716 } 5717 break; 5718 case WebSocketOpcode.binary: 5719 if(m.fin) { 5720 if(onbinarymessage) 5721 onbinarymessage(m.data); 5722 } else { 5723 continuingType = m.opcode; 5724 //continuingDataLength = 0; 5725 continuingData = null; 5726 continuingData ~= m.data; 5727 } 5728 break; 5729 case WebSocketOpcode.close: 5730 readyState_ = CLOSED; 5731 if(onclose) 5732 onclose(); 5733 5734 unregisterActiveSocket(this); 5735 break; 5736 case WebSocketOpcode.ping: 5737 pong(); 5738 break; 5739 case WebSocketOpcode.pong: 5740 // just really references it is still alive, nbd. 5741 break; 5742 default: // ignore though i could and perhaps should throw too 5743 } 5744 } 5745 5746 // the recv thing can be invalidated so gotta copy it over ugh 5747 if(d.length) { 5748 m.data = m.data.dup(); 5749 } 5750 5751 import core.stdc.string; 5752 memmove(receiveBuffer.ptr, d.ptr, d.length); 5753 receiveBufferUsedLength = d.length; 5754 5755 return m; 5756 } 5757 5758 private void autoprocess() { 5759 // FIXME 5760 do { 5761 processOnce(); 5762 } while(lowLevelReceive()); 5763 } 5764 5765 5766 void delegate() onclose; /// 5767 void delegate() onerror; /// 5768 void delegate(in char[]) ontextmessage; /// 5769 void delegate(in ubyte[]) onbinarymessage; /// 5770 void delegate() onopen; /// 5771 5772 /++ 5773 5774 +/ 5775 /// Group: browser_api 5776 void onmessage(void delegate(in char[]) dg) { 5777 ontextmessage = dg; 5778 } 5779 5780 /// ditto 5781 void onmessage(void delegate(in ubyte[]) dg) { 5782 onbinarymessage = dg; 5783 } 5784 5785 /* } end copy/paste */ 5786 5787 5788 } 5789 5790 bool websocketRequested(Cgi cgi) { 5791 return 5792 "sec-websocket-key" in cgi.requestHeaders 5793 && 5794 "connection" in cgi.requestHeaders && 5795 cgi.requestHeaders["connection"].asLowerCase().canFind("upgrade") 5796 && 5797 "upgrade" in cgi.requestHeaders && 5798 cgi.requestHeaders["upgrade"].asLowerCase().equal("websocket") 5799 ; 5800 } 5801 5802 WebSocket acceptWebsocket(Cgi cgi) { 5803 assert(!cgi.closed); 5804 assert(!cgi.outputtedResponseData); 5805 cgi.setResponseStatus("101 Switching Protocols"); 5806 cgi.header("Upgrade: WebSocket"); 5807 cgi.header("Connection: upgrade"); 5808 5809 string key = cgi.requestHeaders["sec-websocket-key"]; 5810 key ~= "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"; // the defined guid from the websocket spec 5811 5812 import std.digest.sha; 5813 auto hash = sha1Of(key); 5814 auto accept = Base64.encode(hash); 5815 5816 cgi.header(("Sec-WebSocket-Accept: " ~ accept).idup); 5817 5818 cgi.websocketMode = true; 5819 cgi.write(""); 5820 5821 cgi.flush(); 5822 5823 return new WebSocket(cgi); 5824 } 5825 5826 // FIXME get websocket to work on other modes, not just embedded_httpd 5827 5828 /* copy/paste in http2.d { */ 5829 enum WebSocketOpcode : ubyte { 5830 continuation = 0, 5831 text = 1, 5832 binary = 2, 5833 // 3, 4, 5, 6, 7 RESERVED 5834 close = 8, 5835 ping = 9, 5836 pong = 10, 5837 // 11,12,13,14,15 RESERVED 5838 } 5839 5840 public struct WebSocketFrame { 5841 private bool populated; 5842 bool fin; 5843 bool rsv1; 5844 bool rsv2; 5845 bool rsv3; 5846 WebSocketOpcode opcode; // 4 bits 5847 bool masked; 5848 ubyte lengthIndicator; // don't set this when building one to send 5849 ulong realLength; // don't use when sending 5850 ubyte[4] maskingKey; // don't set this when sending 5851 ubyte[] data; 5852 5853 static WebSocketFrame simpleMessage(WebSocketOpcode opcode, void[] data) { 5854 WebSocketFrame msg; 5855 msg.fin = true; 5856 msg.opcode = opcode; 5857 msg.data = cast(ubyte[]) data; 5858 5859 return msg; 5860 } 5861 5862 private void send(scope void delegate(ubyte[]) llsend) { 5863 ubyte[64] headerScratch; 5864 int headerScratchPos = 0; 5865 5866 realLength = data.length; 5867 5868 { 5869 ubyte b1; 5870 b1 |= cast(ubyte) opcode; 5871 b1 |= rsv3 ? (1 << 4) : 0; 5872 b1 |= rsv2 ? (1 << 5) : 0; 5873 b1 |= rsv1 ? (1 << 6) : 0; 5874 b1 |= fin ? (1 << 7) : 0; 5875 5876 headerScratch[0] = b1; 5877 headerScratchPos++; 5878 } 5879 5880 { 5881 headerScratchPos++; // we'll set header[1] at the end of this 5882 auto rlc = realLength; 5883 ubyte b2; 5884 b2 |= masked ? (1 << 7) : 0; 5885 5886 assert(headerScratchPos == 2); 5887 5888 if(realLength > 65535) { 5889 // use 64 bit length 5890 b2 |= 0x7f; 5891 5892 // FIXME: double check endinaness 5893 foreach(i; 0 .. 8) { 5894 headerScratch[2 + 7 - i] = rlc & 0x0ff; 5895 rlc >>>= 8; 5896 } 5897 5898 headerScratchPos += 8; 5899 } else if(realLength > 125) { 5900 // use 16 bit length 5901 b2 |= 0x7e; 5902 5903 // FIXME: double check endinaness 5904 foreach(i; 0 .. 2) { 5905 headerScratch[2 + 1 - i] = rlc & 0x0ff; 5906 rlc >>>= 8; 5907 } 5908 5909 headerScratchPos += 2; 5910 } else { 5911 // use 7 bit length 5912 b2 |= realLength & 0b_0111_1111; 5913 } 5914 5915 headerScratch[1] = b2; 5916 } 5917 5918 //assert(!masked, "masking key not properly implemented"); 5919 if(masked) { 5920 // FIXME: randomize this 5921 headerScratch[headerScratchPos .. headerScratchPos + 4] = maskingKey[]; 5922 headerScratchPos += 4; 5923 5924 // we'll just mask it in place... 5925 int keyIdx = 0; 5926 foreach(i; 0 .. data.length) { 5927 data[i] = data[i] ^ maskingKey[keyIdx]; 5928 if(keyIdx == 3) 5929 keyIdx = 0; 5930 else 5931 keyIdx++; 5932 } 5933 } 5934 5935 //writeln("SENDING ", headerScratch[0 .. headerScratchPos], data); 5936 llsend(headerScratch[0 .. headerScratchPos]); 5937 llsend(data); 5938 } 5939 5940 static WebSocketFrame read(ref ubyte[] d) { 5941 WebSocketFrame msg; 5942 5943 auto orig = d; 5944 5945 WebSocketFrame needsMoreData() { 5946 d = orig; 5947 return WebSocketFrame.init; 5948 } 5949 5950 if(d.length < 2) 5951 return needsMoreData(); 5952 5953 ubyte b = d[0]; 5954 5955 msg.populated = true; 5956 5957 msg.opcode = cast(WebSocketOpcode) (b & 0x0f); 5958 b >>= 4; 5959 msg.rsv3 = b & 0x01; 5960 b >>= 1; 5961 msg.rsv2 = b & 0x01; 5962 b >>= 1; 5963 msg.rsv1 = b & 0x01; 5964 b >>= 1; 5965 msg.fin = b & 0x01; 5966 5967 b = d[1]; 5968 msg.masked = (b & 0b1000_0000) ? true : false; 5969 msg.lengthIndicator = b & 0b0111_1111; 5970 5971 d = d[2 .. $]; 5972 5973 if(msg.lengthIndicator == 0x7e) { 5974 // 16 bit length 5975 msg.realLength = 0; 5976 5977 if(d.length < 2) return needsMoreData(); 5978 5979 foreach(i; 0 .. 2) { 5980 msg.realLength |= d[0] << ((1-i) * 8); 5981 d = d[1 .. $]; 5982 } 5983 } else if(msg.lengthIndicator == 0x7f) { 5984 // 64 bit length 5985 msg.realLength = 0; 5986 5987 if(d.length < 8) return needsMoreData(); 5988 5989 foreach(i; 0 .. 8) { 5990 msg.realLength |= d[0] << ((7-i) * 8); 5991 d = d[1 .. $]; 5992 } 5993 } else { 5994 // 7 bit length 5995 msg.realLength = msg.lengthIndicator; 5996 } 5997 5998 if(msg.masked) { 5999 6000 if(d.length < 4) return needsMoreData(); 6001 6002 msg.maskingKey = d[0 .. 4]; 6003 d = d[4 .. $]; 6004 } 6005 6006 if(msg.realLength > d.length) { 6007 return needsMoreData(); 6008 } 6009 6010 msg.data = d[0 .. cast(size_t) msg.realLength]; 6011 d = d[cast(size_t) msg.realLength .. $]; 6012 6013 return msg; 6014 } 6015 6016 void unmaskInPlace() { 6017 if(this.masked) { 6018 int keyIdx = 0; 6019 foreach(i; 0 .. this.data.length) { 6020 this.data[i] = this.data[i] ^ this.maskingKey[keyIdx]; 6021 if(keyIdx == 3) 6022 keyIdx = 0; 6023 else 6024 keyIdx++; 6025 } 6026 } 6027 } 6028 6029 char[] textData() { 6030 return cast(char[]) data; 6031 } 6032 } 6033 /* } */ 6034 } 6035 6036 6037 version(Windows) 6038 { 6039 version(CRuntime_DigitalMars) 6040 { 6041 extern(C) int setmode(int, int) nothrow @nogc; 6042 } 6043 else version(CRuntime_Microsoft) 6044 { 6045 extern(C) int _setmode(int, int) nothrow @nogc; 6046 alias setmode = _setmode; 6047 } 6048 else static assert(0); 6049 } 6050 6051 version(Posix) { 6052 version(CRuntime_Musl) {} else { 6053 import core.sys.posix.unistd; 6054 private extern(C) int posix_spawn(pid_t*, const char*, void*, void*, const char**, const char**); 6055 } 6056 } 6057 6058 6059 // FIXME: these aren't quite public yet. 6060 //private: 6061 6062 // template for laziness 6063 void startAddonServer()(string arg) { 6064 version(OSX) { 6065 assert(0, "Not implemented"); 6066 } else version(linux) { 6067 import core.sys.posix.unistd; 6068 pid_t pid; 6069 const(char)*[16] args; 6070 args[0] = "ARSD_CGI_ADDON_SERVER"; 6071 args[1] = arg.ptr; 6072 posix_spawn(&pid, "/proc/self/exe", 6073 null, 6074 null, 6075 args.ptr, 6076 null // env 6077 ); 6078 } else version(Windows) { 6079 wchar[2048] filename; 6080 auto len = GetModuleFileNameW(null, filename.ptr, cast(DWORD) filename.length); 6081 if(len == 0 || len == filename.length) 6082 throw new Exception("could not get process name to start helper server"); 6083 6084 STARTUPINFOW startupInfo; 6085 startupInfo.cb = cast(DWORD) startupInfo.sizeof; 6086 PROCESS_INFORMATION processInfo; 6087 6088 import std.utf; 6089 6090 // I *MIGHT* need to run it as a new job or a service... 6091 auto ret = CreateProcessW( 6092 filename.ptr, 6093 toUTF16z(arg), 6094 null, // process attributes 6095 null, // thread attributes 6096 false, // inherit handles 6097 0, // creation flags 6098 null, // environment 6099 null, // working directory 6100 &startupInfo, 6101 &processInfo 6102 ); 6103 6104 if(!ret) 6105 throw new Exception("create process failed"); 6106 6107 // when done with those, if we set them 6108 /* 6109 CloseHandle(hStdInput); 6110 CloseHandle(hStdOutput); 6111 CloseHandle(hStdError); 6112 */ 6113 6114 } else static assert(0, "Websocket server not implemented on this system yet (email me, i can prolly do it if you need it)"); 6115 } 6116 6117 // template for laziness 6118 /* 6119 The websocket server is a single-process, single-thread, event 6120 I/O thing. It is passed websockets from other CGI processes 6121 and is then responsible for handling their messages and responses. 6122 Note that the CGI process is responsible for websocket setup, 6123 including authentication, etc. 6124 6125 It also gets data sent to it by other processes and is responsible 6126 for distributing that, as necessary. 6127 */ 6128 void runWebsocketServer()() { 6129 assert(0, "not implemented"); 6130 } 6131 6132 void sendToWebsocketServer(WebSocket ws, string group) { 6133 assert(0, "not implemented"); 6134 } 6135 6136 void sendToWebsocketServer(string content, string group) { 6137 assert(0, "not implemented"); 6138 } 6139 6140 6141 void runEventServer()() { 6142 runAddonServer("/tmp/arsd_cgi_event_server", new EventSourceServerImplementation()); 6143 } 6144 6145 void runTimerServer()() { 6146 runAddonServer("/tmp/arsd_scheduled_job_server", new ScheduledJobServerImplementation()); 6147 } 6148 6149 version(Posix) { 6150 alias LocalServerConnectionHandle = int; 6151 alias CgiConnectionHandle = int; 6152 alias SocketConnectionHandle = int; 6153 6154 enum INVALID_CGI_CONNECTION_HANDLE = -1; 6155 } else version(Windows) { 6156 alias LocalServerConnectionHandle = HANDLE; 6157 version(embedded_httpd_threads) { 6158 alias CgiConnectionHandle = SOCKET; 6159 enum INVALID_CGI_CONNECTION_HANDLE = INVALID_SOCKET; 6160 } else version(fastcgi) { 6161 alias CgiConnectionHandle = void*; // Doesn't actually work! But I don't want compile to fail pointlessly at this point. 6162 enum INVALID_CGI_CONNECTION_HANDLE = null; 6163 } else version(scgi) { 6164 alias CgiConnectionHandle = SOCKET; 6165 enum INVALID_CGI_CONNECTION_HANDLE = INVALID_SOCKET; 6166 } else { /* version(plain_cgi) */ 6167 alias CgiConnectionHandle = HANDLE; 6168 enum INVALID_CGI_CONNECTION_HANDLE = null; 6169 } 6170 alias SocketConnectionHandle = SOCKET; 6171 } 6172 6173 version(with_addon_servers_connections) 6174 LocalServerConnectionHandle openLocalServerConnection()(string name, string arg) { 6175 version(Posix) { 6176 import core.sys.posix.unistd; 6177 import core.sys.posix.sys.un; 6178 6179 int sock = socket(AF_UNIX, SOCK_STREAM, 0); 6180 if(sock == -1) 6181 throw new Exception("socket " ~ to!string(errno)); 6182 6183 scope(failure) 6184 close(sock); 6185 6186 cloexec(sock); 6187 6188 // add-on server processes are assumed to be local, and thus will 6189 // use unix domain sockets. Besides, I want to pass sockets to them, 6190 // so it basically must be local (except for the session server, but meh). 6191 sockaddr_un addr; 6192 addr.sun_family = AF_UNIX; 6193 version(linux) { 6194 // on linux, we will use the abstract namespace 6195 addr.sun_path[0] = 0; 6196 addr.sun_path[1 .. name.length + 1] = cast(typeof(addr.sun_path[])) name[]; 6197 } else { 6198 // but otherwise, just use a file cuz we must. 6199 addr.sun_path[0 .. name.length] = cast(typeof(addr.sun_path[])) name[]; 6200 } 6201 6202 bool alreadyTried; 6203 6204 try_again: 6205 6206 if(connect(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) { 6207 if(!alreadyTried && errno == ECONNREFUSED) { 6208 // try auto-spawning the server, then attempt connection again 6209 startAddonServer(arg); 6210 import core.thread; 6211 Thread.sleep(50.msecs); 6212 alreadyTried = true; 6213 goto try_again; 6214 } else 6215 throw new Exception("connect " ~ to!string(errno)); 6216 } 6217 6218 return sock; 6219 } else version(Windows) { 6220 return null; // FIXME 6221 } 6222 } 6223 6224 version(with_addon_servers_connections) 6225 void closeLocalServerConnection(LocalServerConnectionHandle handle) { 6226 version(Posix) { 6227 import core.sys.posix.unistd; 6228 close(handle); 6229 } else version(Windows) 6230 CloseHandle(handle); 6231 } 6232 6233 void runSessionServer()() { 6234 runAddonServer("/tmp/arsd_session_server", new BasicDataServerImplementation()); 6235 } 6236 6237 version(Posix) 6238 private void makeNonBlocking(int fd) { 6239 import core.sys.posix.fcntl; 6240 auto flags = fcntl(fd, F_GETFL, 0); 6241 if(flags == -1) 6242 throw new Exception("fcntl get"); 6243 flags |= O_NONBLOCK; 6244 auto s = fcntl(fd, F_SETFL, flags); 6245 if(s == -1) 6246 throw new Exception("fcntl set"); 6247 } 6248 6249 import core.stdc.errno; 6250 6251 struct IoOp { 6252 @disable this(); 6253 @disable this(this); 6254 6255 /* 6256 So we want to be able to eventually handle generic sockets too. 6257 */ 6258 6259 enum Read = 1; 6260 enum Write = 2; 6261 enum Accept = 3; 6262 enum ReadSocketHandle = 4; 6263 6264 // Your handler may be called in a different thread than the one that initiated the IO request! 6265 // It is also possible to have multiple io requests being called simultaneously. Use proper thread safety caution. 6266 private bool delegate(IoOp*, int) handler; // returns true if you are done and want it to be closed 6267 private void delegate(IoOp*) closeHandler; 6268 private void delegate(IoOp*) completeHandler; 6269 private int internalFd; 6270 private int operation; 6271 private int bufferLengthAllocated; 6272 private int bufferLengthUsed; 6273 private ubyte[1] internalBuffer; // it can be overallocated! 6274 6275 ubyte[] allocatedBuffer() return { 6276 return internalBuffer.ptr[0 .. bufferLengthAllocated]; 6277 } 6278 6279 ubyte[] usedBuffer() return { 6280 return allocatedBuffer[0 .. bufferLengthUsed]; 6281 } 6282 6283 void reset() { 6284 bufferLengthUsed = 0; 6285 } 6286 6287 int fd() { 6288 return internalFd; 6289 } 6290 } 6291 6292 IoOp* allocateIoOp(int fd, int operation, int bufferSize, bool delegate(IoOp*, int) handler) { 6293 import core.stdc.stdlib; 6294 6295 auto ptr = calloc(IoOp.sizeof + bufferSize, 1); 6296 if(ptr is null) 6297 assert(0); // out of memory! 6298 6299 auto op = cast(IoOp*) ptr; 6300 6301 op.handler = handler; 6302 op.internalFd = fd; 6303 op.operation = operation; 6304 op.bufferLengthAllocated = bufferSize; 6305 op.bufferLengthUsed = 0; 6306 6307 import core.memory; 6308 6309 GC.addRoot(ptr); 6310 6311 return op; 6312 } 6313 6314 void freeIoOp(ref IoOp* ptr) { 6315 6316 import core.memory; 6317 GC.removeRoot(ptr); 6318 6319 import core.stdc.stdlib; 6320 free(ptr); 6321 ptr = null; 6322 } 6323 6324 version(Posix) 6325 version(with_addon_servers_connections) 6326 void nonBlockingWrite(EventIoServer eis, int connection, const void[] data) { 6327 import core.sys.posix.unistd; 6328 6329 auto ret = write(connection, data.ptr, data.length); 6330 if(ret != data.length) { 6331 if(ret == 0 || errno == EPIPE) { 6332 // the file is closed, remove it 6333 eis.fileClosed(connection); 6334 } else 6335 throw new Exception("alas " ~ to!string(ret) ~ " " ~ to!string(errno)); // FIXME 6336 } 6337 } 6338 version(Windows) 6339 version(with_addon_servers_connections) 6340 void nonBlockingWrite(EventIoServer eis, int connection, const void[] data) { 6341 // FIXME 6342 } 6343 6344 bool isInvalidHandle(CgiConnectionHandle h) { 6345 return h == INVALID_CGI_CONNECTION_HANDLE; 6346 } 6347 6348 /+ 6349 https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsarecv 6350 https://support.microsoft.com/en-gb/help/181611/socket-overlapped-i-o-versus-blocking-nonblocking-mode 6351 https://stackoverflow.com/questions/18018489/should-i-use-iocps-or-overlapped-wsasend-receive 6352 https://docs.microsoft.com/en-us/windows/desktop/fileio/i-o-completion-ports 6353 https://docs.microsoft.com/en-us/windows/desktop/fileio/createiocompletionport 6354 https://docs.microsoft.com/en-us/windows/desktop/api/mswsock/nf-mswsock-acceptex 6355 https://docs.microsoft.com/en-us/windows/desktop/Sync/waitable-timer-objects 6356 https://docs.microsoft.com/en-us/windows/desktop/api/synchapi/nf-synchapi-setwaitabletimer 6357 https://docs.microsoft.com/en-us/windows/desktop/Sync/using-a-waitable-timer-with-an-asynchronous-procedure-call 6358 https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsagetoverlappedresult 6359 6360 +/ 6361 6362 /++ 6363 You can customize your server by subclassing the appropriate server. Then, register your 6364 subclass at compile time with the [registerEventIoServer] template, or implement your own 6365 main function and call it yourself. 6366 6367 $(TIP If you make your subclass a `final class`, there is a slight performance improvement.) 6368 +/ 6369 version(with_addon_servers_connections) 6370 interface EventIoServer { 6371 bool handleLocalConnectionData(IoOp* op, int receivedFd); 6372 void handleLocalConnectionClose(IoOp* op); 6373 void handleLocalConnectionComplete(IoOp* op); 6374 void wait_timeout(); 6375 void fileClosed(int fd); 6376 6377 void epoll_fd(int fd); 6378 } 6379 6380 // the sink should buffer it 6381 private void serialize(T)(scope void delegate(ubyte[]) sink, T t) { 6382 static if(is(T == struct)) { 6383 foreach(member; __traits(allMembers, T)) 6384 serialize(sink, __traits(getMember, t, member)); 6385 } else static if(is(T : int)) { 6386 // no need to think of endianness just because this is only used 6387 // for local, same-machine stuff anyway. thanks private lol 6388 sink((cast(ubyte*) &t)[0 .. t.sizeof]); 6389 } else static if(is(T == string) || is(T : const(ubyte)[])) { 6390 // these are common enough to optimize 6391 int len = cast(int) t.length; // want length consistent size tho, in case 32 bit program sends to 64 bit server, etc. 6392 sink((cast(ubyte*) &len)[0 .. int.sizeof]); 6393 sink(cast(ubyte[]) t[]); 6394 } else static if(is(T : A[], A)) { 6395 // generic array is less optimal but still prolly ok 6396 int len = cast(int) t.length; 6397 sink((cast(ubyte*) &len)[0 .. int.sizeof]); 6398 foreach(item; t) 6399 serialize(sink, item); 6400 } else static assert(0, T.stringof); 6401 } 6402 6403 // all may be stack buffers, so use cautio 6404 private void deserialize(T)(scope ubyte[] delegate(int sz) get, scope void delegate(T) dg) { 6405 static if(is(T == struct)) { 6406 T t; 6407 foreach(member; __traits(allMembers, T)) 6408 deserialize!(typeof(__traits(getMember, T, member)))(get, (mbr) { __traits(getMember, t, member) = mbr; }); 6409 dg(t); 6410 } else static if(is(T : int)) { 6411 // no need to think of endianness just because this is only used 6412 // for local, same-machine stuff anyway. thanks private lol 6413 T t; 6414 auto data = get(t.sizeof); 6415 t = (cast(T[]) data)[0]; 6416 dg(t); 6417 } else static if(is(T == string) || is(T : const(ubyte)[])) { 6418 // these are common enough to optimize 6419 int len; 6420 auto data = get(len.sizeof); 6421 len = (cast(int[]) data)[0]; 6422 6423 /* 6424 typeof(T[0])[2000] stackBuffer; 6425 T buffer; 6426 6427 if(len < stackBuffer.length) 6428 buffer = stackBuffer[0 .. len]; 6429 else 6430 buffer = new T(len); 6431 6432 data = get(len * typeof(T[0]).sizeof); 6433 */ 6434 6435 T t = cast(T) get(len * cast(int) typeof(T.init[0]).sizeof); 6436 6437 dg(t); 6438 } else static if(is(T == E[], E)) { 6439 T t; 6440 int len; 6441 auto data = get(len.sizeof); 6442 len = (cast(int[]) data)[0]; 6443 t.length = len; 6444 foreach(ref e; t) { 6445 deserialize!E(get, (ele) { e = ele; }); 6446 } 6447 dg(t); 6448 } else static assert(0, T.stringof); 6449 } 6450 6451 unittest { 6452 serialize((ubyte[] b) { 6453 deserialize!int( sz => b[0 .. sz], (t) { assert(t == 1); }); 6454 }, 1); 6455 serialize((ubyte[] b) { 6456 deserialize!int( sz => b[0 .. sz], (t) { assert(t == 56674); }); 6457 }, 56674); 6458 ubyte[1000] buffer; 6459 int bufferPoint; 6460 void add(ubyte[] b) { 6461 buffer[bufferPoint .. bufferPoint + b.length] = b[]; 6462 bufferPoint += b.length; 6463 } 6464 ubyte[] get(int sz) { 6465 auto b = buffer[bufferPoint .. bufferPoint + sz]; 6466 bufferPoint += sz; 6467 return b; 6468 } 6469 serialize(&add, "test here"); 6470 bufferPoint = 0; 6471 deserialize!string(&get, (t) { assert(t == "test here"); }); 6472 bufferPoint = 0; 6473 6474 struct Foo { 6475 int a; 6476 ubyte c; 6477 string d; 6478 } 6479 serialize(&add, Foo(403, 37, "amazing")); 6480 bufferPoint = 0; 6481 deserialize!Foo(&get, (t) { 6482 assert(t.a == 403); 6483 assert(t.c == 37); 6484 assert(t.d == "amazing"); 6485 }); 6486 bufferPoint = 0; 6487 } 6488 6489 /* 6490 Here's the way the RPC interface works: 6491 6492 You define the interface that lists the functions you can call on the remote process. 6493 The interface may also have static methods for convenience. These forward to a singleton 6494 instance of an auto-generated class, which actually sends the args over the pipe. 6495 6496 An impl class actually implements it. A receiving server deserializes down the pipe and 6497 calls methods on the class. 6498 6499 I went with the interface to get some nice compiler checking and documentation stuff. 6500 6501 I could have skipped the interface and just implemented it all from the server class definition 6502 itself, but then the usage may call the method instead of rpcing it; I just like having the user 6503 interface and the implementation separate so you aren't tempted to `new impl` to call the methods. 6504 6505 6506 I fiddled with newlines in the mixin string to ensure the assert line numbers matched up to the source code line number. Idk why dmd didn't do this automatically, but it was important to me. 6507 6508 Realistically though the bodies would just be 6509 connection.call(this.mangleof, args...) sooooo. 6510 6511 FIXME: overloads aren't supported 6512 */ 6513 6514 /// Base for storing sessions in an array. Exists primarily for internal purposes and you should generally not use this. 6515 interface SessionObject {} 6516 6517 private immutable void delegate(string[])[string] scheduledJobHandlers; 6518 private immutable void delegate(string[])[string] websocketServers; 6519 6520 version(with_breaking_cgi_features) 6521 mixin(q{ 6522 6523 mixin template ImplementRpcClientInterface(T, string serverPath, string cmdArg) { 6524 static import std.traits; 6525 6526 // derivedMembers on an interface seems to give exactly what I want: the virtual functions we need to implement. so I am just going to use it directly without more filtering. 6527 static foreach(idx, member; __traits(derivedMembers, T)) { 6528 static if(__traits(isVirtualFunction, __traits(getMember, T, member))) 6529 mixin( q{ 6530 std.traits.ReturnType!(__traits(getMember, T, member)) 6531 } ~ member ~ q{(std.traits.Parameters!(__traits(getMember, T, member)) params) 6532 { 6533 SerializationBuffer buffer; 6534 auto i = cast(ushort) idx; 6535 serialize(&buffer.sink, i); 6536 serialize(&buffer.sink, __traits(getMember, T, member).mangleof); 6537 foreach(param; params) 6538 serialize(&buffer.sink, param); 6539 6540 auto sendable = buffer.sendable; 6541 6542 version(Posix) {{ 6543 auto ret = send(connectionHandle, sendable.ptr, sendable.length, 0); 6544 6545 if(ret == -1) { 6546 throw new Exception("send returned -1, errno: " ~ to!string(errno)); 6547 } else if(ret == 0) { 6548 throw new Exception("Connection to addon server lost"); 6549 } if(ret < sendable.length) 6550 throw new Exception("Send failed to send all"); 6551 assert(ret == sendable.length); 6552 }} // FIXME Windows impl 6553 6554 static if(!is(typeof(return) == void)) { 6555 // there is a return value; we need to wait for it too 6556 version(Posix) { 6557 ubyte[3000] revBuffer; 6558 auto ret = recv(connectionHandle, revBuffer.ptr, revBuffer.length, 0); 6559 auto got = revBuffer[0 .. ret]; 6560 6561 int dataLocation; 6562 ubyte[] grab(int sz) { 6563 auto d = got[dataLocation .. dataLocation + sz]; 6564 dataLocation += sz; 6565 return d; 6566 } 6567 6568 typeof(return) retu; 6569 deserialize!(typeof(return))(&grab, (a) { retu = a; }); 6570 return retu; 6571 } else { 6572 // FIXME Windows impl 6573 return typeof(return).init; 6574 } 6575 6576 } 6577 }}); 6578 } 6579 6580 private static typeof(this) singletonInstance; 6581 private LocalServerConnectionHandle connectionHandle; 6582 6583 static typeof(this) connection() { 6584 if(singletonInstance is null) { 6585 singletonInstance = new typeof(this)(); 6586 singletonInstance.connect(); 6587 } 6588 return singletonInstance; 6589 } 6590 6591 void connect() { 6592 connectionHandle = openLocalServerConnection(serverPath, cmdArg); 6593 } 6594 6595 void disconnect() { 6596 closeLocalServerConnection(connectionHandle); 6597 } 6598 } 6599 6600 void dispatchRpcServer(Interface, Class)(Class this_, ubyte[] data, int fd) if(is(Class : Interface)) { 6601 ushort calledIdx; 6602 string calledFunction; 6603 6604 int dataLocation; 6605 ubyte[] grab(int sz) { 6606 if(sz == 0) assert(0); 6607 auto d = data[dataLocation .. dataLocation + sz]; 6608 dataLocation += sz; 6609 return d; 6610 } 6611 6612 again: 6613 6614 deserialize!ushort(&grab, (a) { calledIdx = a; }); 6615 deserialize!string(&grab, (a) { calledFunction = a; }); 6616 6617 import std.traits; 6618 6619 sw: switch(calledIdx) { 6620 foreach(idx, memberName; __traits(derivedMembers, Interface)) 6621 static if(__traits(isVirtualFunction, __traits(getMember, Interface, memberName))) { 6622 case idx: 6623 assert(calledFunction == __traits(getMember, Interface, memberName).mangleof); 6624 6625 Parameters!(__traits(getMember, Interface, memberName)) params; 6626 foreach(ref param; params) 6627 deserialize!(typeof(param))(&grab, (a) { param = a; }); 6628 6629 static if(is(ReturnType!(__traits(getMember, Interface, memberName)) == void)) { 6630 __traits(getMember, this_, memberName)(params); 6631 } else { 6632 auto ret = __traits(getMember, this_, memberName)(params); 6633 SerializationBuffer buffer; 6634 serialize(&buffer.sink, ret); 6635 6636 auto sendable = buffer.sendable; 6637 6638 version(Posix) { 6639 auto r = send(fd, sendable.ptr, sendable.length, 0); 6640 if(r == -1) { 6641 throw new Exception("send returned -1, errno: " ~ to!string(errno)); 6642 } else if(r == 0) { 6643 throw new Exception("Connection to addon client lost"); 6644 } if(r < sendable.length) 6645 throw new Exception("Send failed to send all"); 6646 6647 } // FIXME Windows impl 6648 } 6649 break sw; 6650 } 6651 default: assert(0); 6652 } 6653 6654 if(dataLocation != data.length) 6655 goto again; 6656 } 6657 6658 6659 private struct SerializationBuffer { 6660 ubyte[2048] bufferBacking; 6661 int bufferLocation; 6662 void sink(scope ubyte[] data) { 6663 bufferBacking[bufferLocation .. bufferLocation + data.length] = data[]; 6664 bufferLocation += data.length; 6665 } 6666 6667 ubyte[] sendable() return { 6668 return bufferBacking[0 .. bufferLocation]; 6669 } 6670 } 6671 6672 /* 6673 FIXME: 6674 add a version command line arg 6675 version data in the library 6676 management gui as external program 6677 6678 at server with event_fd for each run 6679 use .mangleof in the at function name 6680 6681 i think the at server will have to: 6682 pipe args to the child 6683 collect child output for logging 6684 get child return value for logging 6685 6686 on windows timers work differently. idk how to best combine with the io stuff. 6687 6688 will have to have dump and restore too, so i can restart without losing stuff. 6689 */ 6690 6691 /++ 6692 A convenience object for talking to the [BasicDataServer] from a higher level. 6693 See: [Cgi.getSessionObject]. 6694 6695 You pass it a `Data` struct describing the data you want saved in the session. 6696 Then, this class will generate getter and setter properties that allow access 6697 to that data. 6698 6699 Note that each load and store will be done as-accessed; it doesn't front-load 6700 mutable data nor does it batch updates out of fear of read-modify-write race 6701 conditions. (In fact, right now it does this for everything, but in the future, 6702 I might batch load `immutable` members of the Data struct.) 6703 6704 At some point in the future, I might also let it do different backends, like 6705 a client-side cookie store too, but idk. 6706 6707 Note that the plain-old-data members of your `Data` struct are wrapped by this 6708 interface via a static foreach to make property functions. 6709 6710 See_Also: [MockSession] 6711 +/ 6712 interface Session(Data) : SessionObject { 6713 @property string sessionId() const; 6714 6715 /++ 6716 Starts a new session. Note that a session is also 6717 implicitly started as soon as you write data to it, 6718 so if you need to alter these parameters from their 6719 defaults, be sure to explicitly call this BEFORE doing 6720 any writes to session data. 6721 6722 Params: 6723 idleLifetime = How long, in seconds, the session 6724 should remain in memory when not being read from 6725 or written to. The default is one day. 6726 6727 NOT IMPLEMENTED 6728 6729 useExtendedLifetimeCookie = The session ID is always 6730 stored in a HTTP cookie, and by default, that cookie 6731 is discarded when the user closes their browser. 6732 6733 But if you set this to true, it will use a non-perishable 6734 cookie for the given idleLifetime. 6735 6736 NOT IMPLEMENTED 6737 +/ 6738 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false); 6739 6740 /++ 6741 Regenerates the session ID and updates the associated 6742 cookie. 6743 6744 This is also your chance to change immutable data 6745 (not yet implemented). 6746 +/ 6747 void regenerateId(); 6748 6749 /++ 6750 Terminates this session, deleting all saved data. 6751 +/ 6752 void terminate(); 6753 6754 /++ 6755 Plain-old-data members of your `Data` struct are wrapped here via 6756 the property getters and setters. 6757 6758 If the member is a non-string array, it returns a magical array proxy 6759 object which allows for atomic appends and replaces via overloaded operators. 6760 You can slice this to get a range representing a $(B const) view of the array. 6761 This is to protect you against read-modify-write race conditions. 6762 +/ 6763 static foreach(memberName; __traits(allMembers, Data)) 6764 static if(is(typeof(__traits(getMember, Data, memberName)))) 6765 mixin(q{ 6766 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout; 6767 @property typeof(__traits(getMember, Data, memberName)) } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value); 6768 }); 6769 6770 } 6771 6772 /++ 6773 An implementation of [Session] that works on real cgi connections utilizing the 6774 [BasicDataServer]. 6775 6776 As opposed to a [MockSession] which is made for testing purposes. 6777 6778 You will not construct one of these directly. See [Cgi.getSessionObject] instead. 6779 +/ 6780 class BasicDataServerSession(Data) : Session!Data { 6781 private Cgi cgi; 6782 private string sessionId_; 6783 6784 public @property string sessionId() const { 6785 return sessionId_; 6786 } 6787 6788 protected @property string sessionId(string s) { 6789 return this.sessionId_ = s; 6790 } 6791 6792 private this(Cgi cgi) { 6793 this.cgi = cgi; 6794 if(auto ptr = "sessionId" in cgi.cookies) 6795 sessionId = (*ptr).length ? *ptr : null; 6796 } 6797 6798 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false) { 6799 assert(sessionId is null); 6800 6801 // FIXME: what if there is a session ID cookie, but no corresponding session on the server? 6802 6803 import std.random, std.conv; 6804 sessionId = to!string(uniform(1, long.max)); 6805 6806 BasicDataServer.connection.createSession(sessionId, idleLifetime); 6807 setCookie(); 6808 } 6809 6810 protected void setCookie() { 6811 cgi.setCookie( 6812 "sessionId", sessionId, 6813 0 /* expiration */, 6814 "/" /* path */, 6815 null /* domain */, 6816 true /* http only */, 6817 cgi.https /* if the session is started on https, keep it there, otherwise, be flexible */); 6818 } 6819 6820 void regenerateId() { 6821 if(sessionId is null) { 6822 start(); 6823 return; 6824 } 6825 import std.random, std.conv; 6826 auto oldSessionId = sessionId; 6827 sessionId = to!string(uniform(1, long.max)); 6828 BasicDataServer.connection.renameSession(oldSessionId, sessionId); 6829 setCookie(); 6830 } 6831 6832 void terminate() { 6833 BasicDataServer.connection.destroySession(sessionId); 6834 sessionId = null; 6835 setCookie(); 6836 } 6837 6838 static foreach(memberName; __traits(allMembers, Data)) 6839 static if(is(typeof(__traits(getMember, Data, memberName)))) 6840 mixin(q{ 6841 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout { 6842 if(sessionId is null) 6843 return typeof(return).init; 6844 6845 import std.traits; 6846 auto v = BasicDataServer.connection.getSessionData(sessionId, fullyQualifiedName!Data ~ "." ~ memberName); 6847 if(v.length == 0) 6848 return typeof(return).init; 6849 import std.conv; 6850 // why this cast? to doesn't like being given an inout argument. so need to do it without that, then 6851 // we need to return it and that needed the cast. It should be fine since we basically respect constness.. 6852 // basically. Assuming the session is POD this should be fine. 6853 return cast(typeof(return)) to!(typeof(__traits(getMember, Data, memberName)))(v); 6854 } 6855 @property typeof(__traits(getMember, Data, memberName)) } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value) { 6856 if(sessionId is null) 6857 start(); 6858 import std.conv; 6859 import std.traits; 6860 BasicDataServer.connection.setSessionData(sessionId, fullyQualifiedName!Data ~ "." ~ memberName, to!string(value)); 6861 return value; 6862 } 6863 }); 6864 } 6865 6866 /++ 6867 A mock object that works like the real session, but doesn't actually interact with any actual database or http connection. 6868 Simply stores the data in its instance members. 6869 +/ 6870 class MockSession(Data) : Session!Data { 6871 pure { 6872 @property string sessionId() const { return "mock"; } 6873 void start(int idleLifetime = 2600 * 24, bool useExtendedLifetimeCookie = false) {} 6874 void regenerateId() {} 6875 void terminate() {} 6876 6877 private Data store_; 6878 6879 static foreach(memberName; __traits(allMembers, Data)) 6880 static if(is(typeof(__traits(getMember, Data, memberName)))) 6881 mixin(q{ 6882 @property inout(typeof(__traits(getMember, Data, memberName))) } ~ memberName ~ q{ () inout { 6883 return __traits(getMember, store_, memberName); 6884 } 6885 @property typeof(__traits(getMember, Data, memberName)) } ~ memberName ~ q{ (typeof(__traits(getMember, Data, memberName)) value) { 6886 return __traits(getMember, store_, memberName) = value; 6887 } 6888 }); 6889 } 6890 } 6891 6892 /++ 6893 Direct interface to the basic data add-on server. You can 6894 typically use [Cgi.getSessionObject] as a more convenient interface. 6895 +/ 6896 version(with_addon_servers_connections) 6897 interface BasicDataServer { 6898 /// 6899 void createSession(string sessionId, int lifetime); 6900 /// 6901 void renewSession(string sessionId, int lifetime); 6902 /// 6903 void destroySession(string sessionId); 6904 /// 6905 void renameSession(string oldSessionId, string newSessionId); 6906 6907 /// 6908 void setSessionData(string sessionId, string dataKey, string dataValue); 6909 /// 6910 string getSessionData(string sessionId, string dataKey); 6911 6912 /// 6913 static BasicDataServerConnection connection() { 6914 return BasicDataServerConnection.connection(); 6915 } 6916 } 6917 6918 version(with_addon_servers_connections) 6919 class BasicDataServerConnection : BasicDataServer { 6920 mixin ImplementRpcClientInterface!(BasicDataServer, "/tmp/arsd_session_server", "--session-server"); 6921 } 6922 6923 version(with_addon_servers) 6924 final class BasicDataServerImplementation : BasicDataServer, EventIoServer { 6925 6926 void createSession(string sessionId, int lifetime) { 6927 sessions[sessionId.idup] = Session(lifetime); 6928 } 6929 void destroySession(string sessionId) { 6930 sessions.remove(sessionId); 6931 } 6932 void renewSession(string sessionId, int lifetime) { 6933 sessions[sessionId].lifetime = lifetime; 6934 } 6935 void renameSession(string oldSessionId, string newSessionId) { 6936 sessions[newSessionId.idup] = sessions[oldSessionId]; 6937 sessions.remove(oldSessionId); 6938 } 6939 void setSessionData(string sessionId, string dataKey, string dataValue) { 6940 if(sessionId !in sessions) 6941 createSession(sessionId, 3600); // FIXME? 6942 sessions[sessionId].values[dataKey.idup] = dataValue.idup; 6943 } 6944 string getSessionData(string sessionId, string dataKey) { 6945 if(auto session = sessionId in sessions) { 6946 if(auto data = dataKey in (*session).values) 6947 return *data; 6948 else 6949 return null; // no such data 6950 6951 } else { 6952 return null; // no session 6953 } 6954 } 6955 6956 6957 protected: 6958 6959 struct Session { 6960 int lifetime; 6961 6962 string[string] values; 6963 } 6964 6965 Session[string] sessions; 6966 6967 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 6968 auto data = op.usedBuffer; 6969 dispatchRpcServer!BasicDataServer(this, data, op.fd); 6970 return false; 6971 } 6972 6973 void handleLocalConnectionClose(IoOp* op) {} // doesn't really matter, this is a fairly stateless go 6974 void handleLocalConnectionComplete(IoOp* op) {} // again, irrelevant 6975 void wait_timeout() {} 6976 void fileClosed(int fd) {} // stateless so irrelevant 6977 void epoll_fd(int fd) {} 6978 } 6979 6980 /++ 6981 See [schedule] to make one of these. You then call one of the methods here to set it up: 6982 6983 --- 6984 schedule!fn(args).at(DateTime(2019, 8, 7, 12, 00, 00)); // run the function at August 7, 2019, 12 noon UTC 6985 schedule!fn(args).delay(6.seconds); // run it after waiting 6 seconds 6986 schedule!fn(args).asap(); // run it in the background as soon as the event loop gets around to it 6987 --- 6988 +/ 6989 version(with_addon_servers_connections) 6990 struct ScheduledJobHelper { 6991 private string func; 6992 private string[] args; 6993 private bool consumed; 6994 6995 private this(string func, string[] args) { 6996 this.func = func; 6997 this.args = args; 6998 } 6999 7000 ~this() { 7001 assert(consumed); 7002 } 7003 7004 /++ 7005 Schedules the job to be run at the given time. 7006 +/ 7007 void at(DateTime when, immutable TimeZone timezone = UTC()) { 7008 consumed = true; 7009 7010 auto conn = ScheduledJobServerConnection.connection; 7011 import std.file; 7012 auto st = SysTime(when, timezone); 7013 auto jobId = conn.scheduleJob(1, cast(int) st.toUnixTime(), thisExePath, func, args); 7014 } 7015 7016 /++ 7017 Schedules the job to run at least after the specified delay. 7018 +/ 7019 void delay(Duration delay) { 7020 consumed = true; 7021 7022 auto conn = ScheduledJobServerConnection.connection; 7023 import std.file; 7024 auto jobId = conn.scheduleJob(0, cast(int) delay.total!"seconds", thisExePath, func, args); 7025 } 7026 7027 /++ 7028 Runs the job in the background ASAP. 7029 7030 $(NOTE It may run in a background thread. Don't segfault!) 7031 +/ 7032 void asap() { 7033 consumed = true; 7034 7035 auto conn = ScheduledJobServerConnection.connection; 7036 import std.file; 7037 auto jobId = conn.scheduleJob(0, 1, thisExePath, func, args); 7038 } 7039 7040 /+ 7041 /++ 7042 Schedules the job to recur on the given pattern. 7043 +/ 7044 void recur(string spec) { 7045 7046 } 7047 +/ 7048 } 7049 7050 /++ 7051 First step to schedule a job on the scheduled job server. 7052 7053 The scheduled job needs to be a top-level function that doesn't read any 7054 variables from outside its arguments because it may be run in a new process, 7055 without any context existing later. 7056 7057 You MUST set details on the returned object to actually do anything! 7058 +/ 7059 template schedule(alias fn, T...) if(is(typeof(fn) == function)) { 7060 /// 7061 ScheduledJobHelper schedule(T args) { 7062 // this isn't meant to ever be called, but instead just to 7063 // get the compiler to type check the arguments passed for us 7064 auto sample = delegate() { 7065 fn(args); 7066 }; 7067 string[] sargs; 7068 foreach(arg; args) 7069 sargs ~= to!string(arg); 7070 return ScheduledJobHelper(fn.mangleof, sargs); 7071 } 7072 7073 shared static this() { 7074 scheduledJobHandlers[fn.mangleof] = delegate(string[] sargs) { 7075 import std.traits; 7076 Parameters!fn args; 7077 foreach(idx, ref arg; args) 7078 arg = to!(typeof(arg))(sargs[idx]); 7079 fn(args); 7080 }; 7081 } 7082 } 7083 7084 /// 7085 interface ScheduledJobServer { 7086 /// Use the [schedule] function for a higher-level interface. 7087 int scheduleJob(int whenIs, int when, string executable, string func, string[] args); 7088 /// 7089 void cancelJob(int jobId); 7090 } 7091 7092 version(with_addon_servers_connections) 7093 class ScheduledJobServerConnection : ScheduledJobServer { 7094 mixin ImplementRpcClientInterface!(ScheduledJobServer, "/tmp/arsd_scheduled_job_server", "--timer-server"); 7095 } 7096 7097 version(with_addon_servers) 7098 final class ScheduledJobServerImplementation : ScheduledJobServer, EventIoServer { 7099 // FIXME: we need to handle SIGCHLD in this somehow 7100 // whenIs is 0 for relative, 1 for absolute 7101 protected int scheduleJob(int whenIs, int when, string executable, string func, string[] args) { 7102 auto nj = nextJobId; 7103 nextJobId++; 7104 7105 version(linux) { 7106 import core.sys.linux.timerfd; 7107 import core.sys.linux.epoll; 7108 import core.sys.posix.unistd; 7109 7110 7111 auto fd = timerfd_create(CLOCK_REALTIME, TFD_NONBLOCK | TFD_CLOEXEC); 7112 if(fd == -1) 7113 throw new Exception("fd timer create failed"); 7114 7115 foreach(ref arg; args) 7116 arg = arg.idup; 7117 auto job = Job(executable.idup, func.idup, .dup(args), fd, nj); 7118 7119 itimerspec value; 7120 value.it_value.tv_sec = when; 7121 value.it_value.tv_nsec = 0; 7122 7123 value.it_interval.tv_sec = 0; 7124 value.it_interval.tv_nsec = 0; 7125 7126 if(timerfd_settime(fd, whenIs == 1 ? TFD_TIMER_ABSTIME : 0, &value, null) == -1) 7127 throw new Exception("couldn't set fd timer"); 7128 7129 auto op = allocateIoOp(fd, IoOp.Read, 16, (IoOp* op, int fd) { 7130 jobs.remove(nj); 7131 epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, null); 7132 close(fd); 7133 7134 7135 spawnProcess([job.executable, "--timed-job", job.func] ~ job.args); 7136 7137 return true; 7138 }); 7139 scope(failure) 7140 freeIoOp(op); 7141 7142 epoll_event ev; 7143 ev.events = EPOLLIN | EPOLLET; 7144 ev.data.ptr = op; 7145 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &ev) == -1) 7146 throw new Exception("epoll_ctl " ~ to!string(errno)); 7147 7148 jobs[nj] = job; 7149 return nj; 7150 } else assert(0); 7151 } 7152 7153 protected void cancelJob(int jobId) { 7154 version(linux) { 7155 auto job = jobId in jobs; 7156 if(job is null) 7157 return; 7158 7159 jobs.remove(jobId); 7160 7161 version(linux) { 7162 import core.sys.linux.timerfd; 7163 import core.sys.linux.epoll; 7164 import core.sys.posix.unistd; 7165 epoll_ctl(epoll_fd, EPOLL_CTL_DEL, job.timerfd, null); 7166 close(job.timerfd); 7167 } 7168 } 7169 jobs.remove(jobId); 7170 } 7171 7172 int nextJobId = 1; 7173 static struct Job { 7174 string executable; 7175 string func; 7176 string[] args; 7177 int timerfd; 7178 int id; 7179 } 7180 Job[int] jobs; 7181 7182 7183 // event io server methods below 7184 7185 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 7186 auto data = op.usedBuffer; 7187 dispatchRpcServer!ScheduledJobServer(this, data, op.fd); 7188 return false; 7189 } 7190 7191 void handleLocalConnectionClose(IoOp* op) {} // doesn't really matter, this is a fairly stateless go 7192 void handleLocalConnectionComplete(IoOp* op) {} // again, irrelevant 7193 void wait_timeout() {} 7194 void fileClosed(int fd) {} // stateless so irrelevant 7195 7196 int epoll_fd_; 7197 void epoll_fd(int fd) {this.epoll_fd_ = fd; } 7198 int epoll_fd() { return epoll_fd_; } 7199 } 7200 7201 /// 7202 version(with_addon_servers_connections) 7203 interface EventSourceServer { 7204 /++ 7205 sends this cgi request to the event server so it will be fed events. You should not do anything else with the cgi object after this. 7206 7207 $(WARNING This API is extremely unstable. I might change it or remove it without notice.) 7208 7209 See_Also: 7210 [sendEvent] 7211 +/ 7212 public static void adoptConnection(Cgi cgi, in char[] eventUrl) { 7213 /* 7214 If lastEventId is missing or empty, you just get new events as they come. 7215 7216 If it is set from something else, it sends all since then (that are still alive) 7217 down the pipe immediately. 7218 7219 The reason it can come from the header is that's what the standard defines for 7220 browser reconnects. The reason it can come from a query string is just convenience 7221 in catching up in a user-defined manner. 7222 7223 The reason the header overrides the query string is if the browser tries to reconnect, 7224 it will send the header AND the query (it reconnects to the same url), so we just 7225 want to do the restart thing. 7226 7227 Note that if you ask for "0" as the lastEventId, it will get ALL still living events. 7228 */ 7229 string lastEventId = cgi.lastEventId; 7230 if(lastEventId.length == 0 && "lastEventId" in cgi.get) 7231 lastEventId = cgi.get["lastEventId"]; 7232 7233 cgi.setResponseContentType("text/event-stream"); 7234 cgi.write(":\n", false); // to initialize the chunking and send headers before keeping the fd for later 7235 cgi.flush(); 7236 7237 cgi.closed = true; 7238 auto s = openLocalServerConnection("/tmp/arsd_cgi_event_server", "--event-server"); 7239 scope(exit) 7240 closeLocalServerConnection(s); 7241 7242 version(fastcgi) 7243 throw new Exception("sending fcgi connections not supported"); 7244 else { 7245 auto fd = cgi.getOutputFileHandle(); 7246 if(isInvalidHandle(fd)) 7247 throw new Exception("bad fd from cgi!"); 7248 7249 EventSourceServerImplementation.SendableEventConnection sec; 7250 sec.populate(cgi.responseChunked, eventUrl, lastEventId); 7251 7252 version(Posix) { 7253 auto res = write_fd(s, cast(void*) &sec, sec.sizeof, fd); 7254 assert(res == sec.sizeof); 7255 } else version(Windows) { 7256 // FIXME 7257 } 7258 } 7259 } 7260 7261 /++ 7262 Sends an event to the event server, starting it if necessary. The event server will distribute it to any listening clients, and store it for `lifetime` seconds for any later listening clients to catch up later. 7263 7264 $(WARNING This API is extremely unstable. I might change it or remove it without notice.) 7265 7266 Params: 7267 url = A string identifying this event "bucket". Listening clients must also connect to this same string. I called it `url` because I envision it being just passed as the url of the request. 7268 event = the event type string, which is used in the Javascript addEventListener API on EventSource 7269 data = the event data. Available in JS as `event.data`. 7270 lifetime = the amount of time to keep this event for replaying on the event server. 7271 7272 See_Also: 7273 [sendEventToEventServer] 7274 +/ 7275 public static void sendEvent(string url, string event, string data, int lifetime) { 7276 auto s = openLocalServerConnection("/tmp/arsd_cgi_event_server", "--event-server"); 7277 scope(exit) 7278 closeLocalServerConnection(s); 7279 7280 EventSourceServerImplementation.SendableEvent sev; 7281 sev.populate(url, event, data, lifetime); 7282 7283 version(Posix) { 7284 auto ret = send(s, &sev, sev.sizeof, 0); 7285 assert(ret == sev.sizeof); 7286 } else version(Windows) { 7287 // FIXME 7288 } 7289 } 7290 7291 /++ 7292 Messages sent to `url` will also be sent to anyone listening on `forwardUrl`. 7293 7294 See_Also: [disconnect] 7295 +/ 7296 void connect(string url, string forwardUrl); 7297 7298 /++ 7299 Disconnects `forwardUrl` from `url` 7300 7301 See_Also: [connect] 7302 +/ 7303 void disconnect(string url, string forwardUrl); 7304 } 7305 7306 /// 7307 version(with_addon_servers) 7308 final class EventSourceServerImplementation : EventSourceServer, EventIoServer { 7309 7310 protected: 7311 7312 void connect(string url, string forwardUrl) { 7313 pipes[url] ~= forwardUrl; 7314 } 7315 void disconnect(string url, string forwardUrl) { 7316 auto t = url in pipes; 7317 if(t is null) 7318 return; 7319 foreach(idx, n; (*t)) 7320 if(n == forwardUrl) { 7321 (*t)[idx] = (*t)[$-1]; 7322 (*t) = (*t)[0 .. $-1]; 7323 break; 7324 } 7325 } 7326 7327 bool handleLocalConnectionData(IoOp* op, int receivedFd) { 7328 if(receivedFd != -1) { 7329 //writeln("GOT FD ", receivedFd, " -- ", op.usedBuffer); 7330 7331 //core.sys.posix.unistd.write(receivedFd, "hello".ptr, 5); 7332 7333 SendableEventConnection* got = cast(SendableEventConnection*) op.usedBuffer.ptr; 7334 7335 auto url = got.url.idup; 7336 eventConnectionsByUrl[url] ~= EventConnection(receivedFd, got.responseChunked > 0 ? true : false); 7337 7338 // FIXME: catch up on past messages here 7339 } else { 7340 auto data = op.usedBuffer; 7341 auto event = cast(SendableEvent*) data.ptr; 7342 7343 if(event.magic == 0xdeadbeef) { 7344 handleInputEvent(event); 7345 7346 if(event.url in pipes) 7347 foreach(pipe; pipes[event.url]) { 7348 event.url = pipe; 7349 handleInputEvent(event); 7350 } 7351 } else { 7352 dispatchRpcServer!EventSourceServer(this, data, op.fd); 7353 } 7354 } 7355 return false; 7356 } 7357 void handleLocalConnectionClose(IoOp* op) {} 7358 void handleLocalConnectionComplete(IoOp* op) {} 7359 7360 void wait_timeout() { 7361 // just keeping alive 7362 foreach(url, connections; eventConnectionsByUrl) 7363 foreach(connection; connections) 7364 if(connection.needsChunking) 7365 nonBlockingWrite(this, connection.fd, "2\r\n:\n"); 7366 else 7367 nonBlockingWrite(this, connection.fd, ":\n"); 7368 } 7369 7370 void fileClosed(int fd) { 7371 outer: foreach(url, ref connections; eventConnectionsByUrl) { 7372 foreach(idx, conn; connections) { 7373 if(fd == conn.fd) { 7374 connections[idx] = connections[$-1]; 7375 connections = connections[0 .. $ - 1]; 7376 continue outer; 7377 } 7378 } 7379 } 7380 } 7381 7382 void epoll_fd(int fd) {} 7383 7384 7385 private: 7386 7387 7388 struct SendableEventConnection { 7389 ubyte responseChunked; 7390 7391 int urlLength; 7392 char[256] urlBuffer = 0; 7393 7394 int lastEventIdLength; 7395 char[32] lastEventIdBuffer = 0; 7396 7397 char[] url() return { 7398 return urlBuffer[0 .. urlLength]; 7399 } 7400 void url(in char[] u) { 7401 urlBuffer[0 .. u.length] = u[]; 7402 urlLength = cast(int) u.length; 7403 } 7404 char[] lastEventId() return { 7405 return lastEventIdBuffer[0 .. lastEventIdLength]; 7406 } 7407 void populate(bool responseChunked, in char[] url, in char[] lastEventId) 7408 in { 7409 assert(url.length < this.urlBuffer.length); 7410 assert(lastEventId.length < this.lastEventIdBuffer.length); 7411 } 7412 do { 7413 this.responseChunked = responseChunked ? 1 : 0; 7414 this.urlLength = cast(int) url.length; 7415 this.lastEventIdLength = cast(int) lastEventId.length; 7416 7417 this.urlBuffer[0 .. url.length] = url[]; 7418 this.lastEventIdBuffer[0 .. lastEventId.length] = lastEventId[]; 7419 } 7420 } 7421 7422 struct SendableEvent { 7423 int magic = 0xdeadbeef; 7424 int urlLength; 7425 char[256] urlBuffer = 0; 7426 int typeLength; 7427 char[32] typeBuffer = 0; 7428 int messageLength; 7429 char[2048] messageBuffer = 0; 7430 int _lifetime; 7431 7432 char[] message() return { 7433 return messageBuffer[0 .. messageLength]; 7434 } 7435 char[] type() return { 7436 return typeBuffer[0 .. typeLength]; 7437 } 7438 char[] url() return { 7439 return urlBuffer[0 .. urlLength]; 7440 } 7441 void url(in char[] u) { 7442 urlBuffer[0 .. u.length] = u[]; 7443 urlLength = cast(int) u.length; 7444 } 7445 int lifetime() { 7446 return _lifetime; 7447 } 7448 7449 /// 7450 void populate(string url, string type, string message, int lifetime) 7451 in { 7452 assert(url.length < this.urlBuffer.length); 7453 assert(type.length < this.typeBuffer.length); 7454 assert(message.length < this.messageBuffer.length); 7455 } 7456 do { 7457 this.urlLength = cast(int) url.length; 7458 this.typeLength = cast(int) type.length; 7459 this.messageLength = cast(int) message.length; 7460 this._lifetime = lifetime; 7461 7462 this.urlBuffer[0 .. url.length] = url[]; 7463 this.typeBuffer[0 .. type.length] = type[]; 7464 this.messageBuffer[0 .. message.length] = message[]; 7465 } 7466 } 7467 7468 struct EventConnection { 7469 int fd; 7470 bool needsChunking; 7471 } 7472 7473 private EventConnection[][string] eventConnectionsByUrl; 7474 private string[][string] pipes; 7475 7476 private void handleInputEvent(scope SendableEvent* event) { 7477 static int eventId; 7478 7479 static struct StoredEvent { 7480 int id; 7481 string type; 7482 string message; 7483 int lifetimeRemaining; 7484 } 7485 7486 StoredEvent[][string] byUrl; 7487 7488 int thisId = ++eventId; 7489 7490 if(event.lifetime) 7491 byUrl[event.url.idup] ~= StoredEvent(thisId, event.type.idup, event.message.idup, event.lifetime); 7492 7493 auto connectionsPtr = event.url in eventConnectionsByUrl; 7494 EventConnection[] connections; 7495 if(connectionsPtr is null) 7496 return; 7497 else 7498 connections = *connectionsPtr; 7499 7500 char[4096] buffer; 7501 char[] formattedMessage; 7502 7503 void append(const char[] a) { 7504 // the 6's here are to leave room for a HTTP chunk header, if it proves necessary 7505 buffer[6 + formattedMessage.length .. 6 + formattedMessage.length + a.length] = a[]; 7506 formattedMessage = buffer[6 .. 6 + formattedMessage.length + a.length]; 7507 } 7508 7509 import std.algorithm.iteration; 7510 7511 if(connections.length) { 7512 append("id: "); 7513 append(to!string(thisId)); 7514 append("\n"); 7515 7516 append("event: "); 7517 append(event.type); 7518 append("\n"); 7519 7520 foreach(line; event.message.splitter("\n")) { 7521 append("data: "); 7522 append(line); 7523 append("\n"); 7524 } 7525 7526 append("\n"); 7527 } 7528 7529 // chunk it for HTTP! 7530 auto len = toHex(formattedMessage.length); 7531 buffer[4 .. 6] = "\r\n"[]; 7532 buffer[4 - len.length .. 4] = len[]; 7533 7534 auto chunkedMessage = buffer[4 - len.length .. 6 + formattedMessage.length]; 7535 // done 7536 7537 // FIXME: send back requests when needed 7538 // FIXME: send a single ":\n" every 15 seconds to keep alive 7539 7540 foreach(connection; connections) { 7541 if(connection.needsChunking) 7542 nonBlockingWrite(this, connection.fd, chunkedMessage); 7543 else 7544 nonBlockingWrite(this, connection.fd, formattedMessage); 7545 } 7546 } 7547 } 7548 7549 void runAddonServer(EIS)(string localListenerName, EIS eis) if(is(EIS : EventIoServer)) { 7550 version(Posix) { 7551 7552 import core.sys.posix.unistd; 7553 import core.sys.posix.fcntl; 7554 import core.sys.posix.sys.un; 7555 7556 import core.sys.posix.signal; 7557 signal(SIGPIPE, SIG_IGN); 7558 7559 static extern(C) void sigchldhandler(int) { 7560 int status; 7561 import w = core.sys.posix.sys.wait; 7562 w.wait(&status); 7563 } 7564 signal(SIGCHLD, &sigchldhandler); 7565 7566 int sock = socket(AF_UNIX, SOCK_STREAM, 0); 7567 if(sock == -1) 7568 throw new Exception("socket " ~ to!string(errno)); 7569 7570 scope(failure) 7571 close(sock); 7572 7573 cloexec(sock); 7574 7575 // add-on server processes are assumed to be local, and thus will 7576 // use unix domain sockets. Besides, I want to pass sockets to them, 7577 // so it basically must be local (except for the session server, but meh). 7578 sockaddr_un addr; 7579 addr.sun_family = AF_UNIX; 7580 version(linux) { 7581 // on linux, we will use the abstract namespace 7582 addr.sun_path[0] = 0; 7583 addr.sun_path[1 .. localListenerName.length + 1] = cast(typeof(addr.sun_path[])) localListenerName[]; 7584 } else { 7585 // but otherwise, just use a file cuz we must. 7586 addr.sun_path[0 .. localListenerName.length] = cast(typeof(addr.sun_path[])) localListenerName[]; 7587 } 7588 7589 if(bind(sock, cast(sockaddr*) &addr, addr.sizeof) == -1) 7590 throw new Exception("bind " ~ to!string(errno)); 7591 7592 if(listen(sock, 128) == -1) 7593 throw new Exception("listen " ~ to!string(errno)); 7594 7595 makeNonBlocking(sock); 7596 7597 version(linux) { 7598 import core.sys.linux.epoll; 7599 auto epoll_fd = epoll_create1(EPOLL_CLOEXEC); 7600 if(epoll_fd == -1) 7601 throw new Exception("epoll_create1 " ~ to!string(errno)); 7602 scope(failure) 7603 close(epoll_fd); 7604 } else { 7605 import core.sys.posix.poll; 7606 } 7607 7608 version(linux) 7609 eis.epoll_fd = epoll_fd; 7610 7611 auto acceptOp = allocateIoOp(sock, IoOp.Read, 0, null); 7612 scope(exit) 7613 freeIoOp(acceptOp); 7614 7615 version(linux) { 7616 epoll_event ev; 7617 ev.events = EPOLLIN | EPOLLET; 7618 ev.data.ptr = acceptOp; 7619 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, sock, &ev) == -1) 7620 throw new Exception("epoll_ctl " ~ to!string(errno)); 7621 7622 epoll_event[64] events; 7623 } else { 7624 pollfd[] pollfds; 7625 IoOp*[int] ioops; 7626 pollfds ~= pollfd(sock, POLLIN); 7627 ioops[sock] = acceptOp; 7628 } 7629 7630 while(true) { 7631 7632 // FIXME: it should actually do a timerfd that runs on any thing that hasn't been run recently 7633 7634 int timeout_milliseconds = 15000; // -1; // infinite 7635 //writeln("waiting for ", name); 7636 7637 version(linux) { 7638 auto nfds = epoll_wait(epoll_fd, events.ptr, events.length, timeout_milliseconds); 7639 if(nfds == -1) { 7640 if(errno == EINTR) 7641 continue; 7642 throw new Exception("epoll_wait " ~ to!string(errno)); 7643 } 7644 } else { 7645 int nfds = poll(pollfds.ptr, cast(int) pollfds.length, timeout_milliseconds); 7646 size_t lastIdx = 0; 7647 } 7648 7649 if(nfds == 0) { 7650 eis.wait_timeout(); 7651 } 7652 7653 foreach(idx; 0 .. nfds) { 7654 version(linux) { 7655 auto flags = events[idx].events; 7656 auto ioop = cast(IoOp*) events[idx].data.ptr; 7657 } else { 7658 IoOp* ioop; 7659 foreach(tidx, thing; pollfds[lastIdx .. $]) { 7660 if(thing.revents) { 7661 ioop = ioops[thing.fd]; 7662 lastIdx += tidx + 1; 7663 break; 7664 } 7665 } 7666 } 7667 7668 //writeln(flags, " ", ioop.fd); 7669 7670 void newConnection() { 7671 // on edge triggering, it is important that we get it all 7672 while(true) { 7673 auto size = cast(uint) addr.sizeof; 7674 auto ns = accept(sock, cast(sockaddr*) &addr, &size); 7675 if(ns == -1) { 7676 if(errno == EAGAIN || errno == EWOULDBLOCK) { 7677 // all done, got it all 7678 break; 7679 } 7680 throw new Exception("accept " ~ to!string(errno)); 7681 } 7682 cloexec(ns); 7683 7684 makeNonBlocking(ns); 7685 auto niop = allocateIoOp(ns, IoOp.ReadSocketHandle, 4096, &eis.handleLocalConnectionData); 7686 niop.closeHandler = &eis.handleLocalConnectionClose; 7687 niop.completeHandler = &eis.handleLocalConnectionComplete; 7688 scope(failure) freeIoOp(niop); 7689 7690 version(linux) { 7691 epoll_event nev; 7692 nev.events = EPOLLIN | EPOLLET; 7693 nev.data.ptr = niop; 7694 if(epoll_ctl(epoll_fd, EPOLL_CTL_ADD, ns, &nev) == -1) 7695 throw new Exception("epoll_ctl " ~ to!string(errno)); 7696 } else { 7697 bool found = false; 7698 foreach(ref pfd; pollfds) { 7699 if(pfd.fd < 0) { 7700 pfd.fd = ns; 7701 found = true; 7702 } 7703 } 7704 if(!found) 7705 pollfds ~= pollfd(ns, POLLIN); 7706 ioops[ns] = niop; 7707 } 7708 } 7709 } 7710 7711 bool newConnectionCondition() { 7712 version(linux) 7713 return ioop.fd == sock && (flags & EPOLLIN); 7714 else 7715 return pollfds[idx].fd == sock && (pollfds[idx].revents & POLLIN); 7716 } 7717 7718 if(newConnectionCondition()) { 7719 newConnection(); 7720 } else if(ioop.operation == IoOp.ReadSocketHandle) { 7721 while(true) { 7722 int in_fd; 7723 auto got = read_fd(ioop.fd, ioop.allocatedBuffer.ptr, ioop.allocatedBuffer.length, &in_fd); 7724 if(got == -1) { 7725 if(errno == EAGAIN || errno == EWOULDBLOCK) { 7726 // all done, got it all 7727 if(ioop.completeHandler) 7728 ioop.completeHandler(ioop); 7729 break; 7730 } 7731 throw new Exception("recv " ~ to!string(errno)); 7732 } 7733 7734 if(got == 0) { 7735 if(ioop.closeHandler) { 7736 ioop.closeHandler(ioop); 7737 version(linux) {} // nothing needed 7738 else { 7739 foreach(ref pfd; pollfds) { 7740 if(pfd.fd == ioop.fd) 7741 pfd.fd = -1; 7742 } 7743 } 7744 } 7745 close(ioop.fd); 7746 freeIoOp(ioop); 7747 break; 7748 } 7749 7750 ioop.bufferLengthUsed = cast(int) got; 7751 ioop.handler(ioop, in_fd); 7752 } 7753 } else if(ioop.operation == IoOp.Read) { 7754 while(true) { 7755 auto got = read(ioop.fd, ioop.allocatedBuffer.ptr, ioop.allocatedBuffer.length); 7756 if(got == -1) { 7757 if(errno == EAGAIN || errno == EWOULDBLOCK) { 7758 // all done, got it all 7759 if(ioop.completeHandler) 7760 ioop.completeHandler(ioop); 7761 break; 7762 } 7763 throw new Exception("recv " ~ to!string(ioop.fd) ~ " errno " ~ to!string(errno)); 7764 } 7765 7766 if(got == 0) { 7767 if(ioop.closeHandler) 7768 ioop.closeHandler(ioop); 7769 close(ioop.fd); 7770 freeIoOp(ioop); 7771 break; 7772 } 7773 7774 ioop.bufferLengthUsed = cast(int) got; 7775 if(ioop.handler(ioop, ioop.fd)) { 7776 close(ioop.fd); 7777 freeIoOp(ioop); 7778 break; 7779 } 7780 } 7781 } 7782 7783 // EPOLLHUP? 7784 } 7785 } 7786 } else version(Windows) { 7787 7788 // set up a named pipe 7789 // https://msdn.microsoft.com/en-us/library/windows/desktop/ms724251(v=vs.85).aspx 7790 // https://docs.microsoft.com/en-us/windows/desktop/api/winsock2/nf-winsock2-wsaduplicatesocketw 7791 // https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-getnamedpipeserverprocessid 7792 7793 } else static assert(0); 7794 } 7795 7796 7797 version(with_sendfd) 7798 // copied from the web and ported from C 7799 // see https://stackoverflow.com/questions/2358684/can-i-share-a-file-descriptor-to-another-process-on-linux-or-are-they-local-to-t 7800 ssize_t write_fd(int fd, void *ptr, size_t nbytes, int sendfd) { 7801 msghdr msg; 7802 iovec[1] iov; 7803 7804 version(OSX) { 7805 //msg.msg_accrights = cast(cattr_t) &sendfd; 7806 //msg.msg_accrightslen = int.sizeof; 7807 } else { 7808 union ControlUnion { 7809 cmsghdr cm; 7810 char[CMSG_SPACE(int.sizeof)] control; 7811 } 7812 7813 ControlUnion control_un; 7814 cmsghdr* cmptr; 7815 7816 msg.msg_control = control_un.control.ptr; 7817 msg.msg_controllen = control_un.control.length; 7818 7819 cmptr = CMSG_FIRSTHDR(&msg); 7820 cmptr.cmsg_len = CMSG_LEN(int.sizeof); 7821 cmptr.cmsg_level = SOL_SOCKET; 7822 cmptr.cmsg_type = SCM_RIGHTS; 7823 *(cast(int *) CMSG_DATA(cmptr)) = sendfd; 7824 } 7825 7826 msg.msg_name = null; 7827 msg.msg_namelen = 0; 7828 7829 iov[0].iov_base = ptr; 7830 iov[0].iov_len = nbytes; 7831 msg.msg_iov = iov.ptr; 7832 msg.msg_iovlen = 1; 7833 7834 return sendmsg(fd, &msg, 0); 7835 } 7836 7837 version(with_sendfd) 7838 // copied from the web and ported from C 7839 ssize_t read_fd(int fd, void *ptr, size_t nbytes, int *recvfd) { 7840 msghdr msg; 7841 iovec[1] iov; 7842 ssize_t n; 7843 int newfd; 7844 7845 version(OSX) { 7846 //msg.msg_accrights = cast(cattr_t) recvfd; 7847 //msg.msg_accrightslen = int.sizeof; 7848 } else { 7849 union ControlUnion { 7850 cmsghdr cm; 7851 char[CMSG_SPACE(int.sizeof)] control; 7852 } 7853 ControlUnion control_un; 7854 cmsghdr* cmptr; 7855 7856 msg.msg_control = control_un.control.ptr; 7857 msg.msg_controllen = control_un.control.length; 7858 } 7859 7860 msg.msg_name = null; 7861 msg.msg_namelen = 0; 7862 7863 iov[0].iov_base = ptr; 7864 iov[0].iov_len = nbytes; 7865 msg.msg_iov = iov.ptr; 7866 msg.msg_iovlen = 1; 7867 7868 if ( (n = recvmsg(fd, &msg, 0)) <= 0) 7869 return n; 7870 7871 version(OSX) { 7872 //if(msg.msg_accrightslen != int.sizeof) 7873 //*recvfd = -1; 7874 } else { 7875 if ( (cmptr = CMSG_FIRSTHDR(&msg)) != null && 7876 cmptr.cmsg_len == CMSG_LEN(int.sizeof)) { 7877 if (cmptr.cmsg_level != SOL_SOCKET) 7878 throw new Exception("control level != SOL_SOCKET"); 7879 if (cmptr.cmsg_type != SCM_RIGHTS) 7880 throw new Exception("control type != SCM_RIGHTS"); 7881 *recvfd = *(cast(int *) CMSG_DATA(cmptr)); 7882 } else 7883 *recvfd = -1; /* descriptor was not passed */ 7884 } 7885 7886 return n; 7887 } 7888 /* end read_fd */ 7889 7890 7891 /* 7892 Event source stuff 7893 7894 The api is: 7895 7896 sendEvent(string url, string type, string data, int timeout = 60*10); 7897 7898 attachEventListener(string url, int fd, lastId) 7899 7900 7901 It just sends to all attached listeners, and stores it until the timeout 7902 for replaying via lastEventId. 7903 */ 7904 7905 /* 7906 Session process stuff 7907 7908 it stores it all. the cgi object has a session object that can grab it 7909 7910 session may be done in the same process if possible, there is a version 7911 switch to choose if you want to override. 7912 */ 7913 7914 struct DispatcherDefinition(alias dispatchHandler, DispatcherDetails = typeof(null)) {// if(is(typeof(dispatchHandler("str", Cgi.init, void) == bool))) { // bool delegate(string urlPrefix, Cgi cgi) dispatchHandler; 7915 alias handler = dispatchHandler; 7916 string urlPrefix; 7917 bool rejectFurther; 7918 immutable(DispatcherDetails) details; 7919 } 7920 7921 private string urlify(string name) pure { 7922 return beautify(name, '-', true); 7923 } 7924 7925 private string beautify(string name, char space = ' ', bool allLowerCase = false) pure { 7926 if(name == "id") 7927 return allLowerCase ? name : "ID"; 7928 7929 char[160] buffer; 7930 int bufferIndex = 0; 7931 bool shouldCap = true; 7932 bool shouldSpace; 7933 bool lastWasCap; 7934 foreach(idx, char ch; name) { 7935 if(bufferIndex == buffer.length) return name; // out of space, just give up, not that important 7936 7937 if((ch >= 'A' && ch <= 'Z') || ch == '_') { 7938 if(lastWasCap) { 7939 // two caps in a row, don't change. Prolly acronym. 7940 } else { 7941 if(idx) 7942 shouldSpace = true; // new word, add space 7943 } 7944 7945 lastWasCap = true; 7946 } else { 7947 lastWasCap = false; 7948 } 7949 7950 if(shouldSpace) { 7951 buffer[bufferIndex++] = space; 7952 if(bufferIndex == buffer.length) return name; // out of space, just give up, not that important 7953 shouldSpace = false; 7954 } 7955 if(shouldCap) { 7956 if(ch >= 'a' && ch <= 'z') 7957 ch -= 32; 7958 shouldCap = false; 7959 } 7960 if(allLowerCase && ch >= 'A' && ch <= 'Z') 7961 ch += 32; 7962 buffer[bufferIndex++] = ch; 7963 } 7964 return buffer[0 .. bufferIndex].idup; 7965 } 7966 7967 /* 7968 string urlFor(alias func)() { 7969 return __traits(identifier, func); 7970 } 7971 */ 7972 7973 /++ 7974 UDA: The name displayed to the user in auto-generated HTML. 7975 7976 Default is `beautify(identifier)`. 7977 +/ 7978 struct DisplayName { 7979 string name; 7980 } 7981 7982 /++ 7983 UDA: The name used in the URL or web parameter. 7984 7985 Default is `urlify(identifier)` for functions and `identifier` for parameters and data members. 7986 +/ 7987 struct UrlName { 7988 string name; 7989 } 7990 7991 /++ 7992 UDA: default format to respond for this method 7993 +/ 7994 struct DefaultFormat { string value; } 7995 7996 class MissingArgumentException : Exception { 7997 string functionName; 7998 string argumentName; 7999 string argumentType; 8000 8001 this(string functionName, string argumentName, string argumentType, string file = __FILE__, size_t line = __LINE__, Throwable next = null) { 8002 this.functionName = functionName; 8003 this.argumentName = argumentName; 8004 this.argumentType = argumentType; 8005 8006 super("Missing Argument: " ~ this.argumentName, file, line, next); 8007 } 8008 } 8009 8010 /++ 8011 This can be attached to any constructor or function called from the cgi system. 8012 8013 If it is present, the function argument can NOT be set from web params, but instead 8014 is set to the return value of the given `func`. 8015 8016 If `func` can take a parameter of type [Cgi], it will be passed the one representing 8017 the current request. Otherwise, it must take zero arguments. 8018 8019 Any params in your function of type `Cgi` are automatically assumed to take the cgi object 8020 for the connection. Any of type [Session] (with an argument) is also assumed to come from 8021 the cgi object. 8022 8023 const arguments are also supported. 8024 +/ 8025 struct ifCalledFromWeb(alias func) {} 8026 8027 // it only looks at query params for GET requests, the rest must be in the body for a function argument. 8028 auto callFromCgi(alias method, T)(T dg, Cgi cgi) { 8029 8030 // FIXME: any array of structs should also be settable or gettable from csv as well. 8031 8032 // FIXME: think more about checkboxes and bools. 8033 8034 import std.traits; 8035 8036 Parameters!method params; 8037 alias idents = ParameterIdentifierTuple!method; 8038 alias defaults = ParameterDefaults!method; 8039 8040 const(string)[] names; 8041 const(string)[] values; 8042 8043 // first, check for missing arguments and initialize to defaults if necessary 8044 8045 static if(is(typeof(method) P == __parameters)) 8046 foreach(idx, param; P) {{ 8047 // see: mustNotBeSetFromWebParams 8048 static if(is(param : Cgi)) { 8049 static assert(!is(param == immutable)); 8050 cast() params[idx] = cgi; 8051 } else static if(is(param == Session!D, D)) { 8052 static assert(!is(param == immutable)); 8053 cast() params[idx] = cgi.getSessionObject!D(); 8054 } else { 8055 bool populated; 8056 foreach(uda; __traits(getAttributes, P[idx .. idx + 1])) { 8057 static if(is(uda == ifCalledFromWeb!func, alias func)) { 8058 static if(is(typeof(func(cgi)))) 8059 params[idx] = func(cgi); 8060 else 8061 params[idx] = func(); 8062 8063 populated = true; 8064 } 8065 } 8066 8067 if(!populated) { 8068 static if(__traits(compiles, { params[idx] = param.getAutomaticallyForCgi(cgi); } )) { 8069 params[idx] = param.getAutomaticallyForCgi(cgi); 8070 populated = true; 8071 } 8072 } 8073 8074 if(!populated) { 8075 auto ident = idents[idx]; 8076 if(cgi.requestMethod == Cgi.RequestMethod.GET) { 8077 if(ident !in cgi.get) { 8078 static if(is(defaults[idx] == void)) { 8079 static if(is(param == bool)) 8080 params[idx] = false; 8081 else 8082 throw new MissingArgumentException(__traits(identifier, method), ident, param.stringof); 8083 } else 8084 params[idx] = defaults[idx]; 8085 } 8086 } else { 8087 if(ident !in cgi.post) { 8088 static if(is(defaults[idx] == void)) { 8089 static if(is(param == bool)) 8090 params[idx] = false; 8091 else 8092 throw new MissingArgumentException(__traits(identifier, method), ident, param.stringof); 8093 } else 8094 params[idx] = defaults[idx]; 8095 } 8096 } 8097 } 8098 } 8099 }} 8100 8101 // second, parse the arguments in order to build up arrays, etc. 8102 8103 static bool setVariable(T)(string name, string paramName, T* what, string value) { 8104 static if(is(T == struct)) { 8105 if(name == paramName) { 8106 *what = T.init; 8107 return true; 8108 } else { 8109 // could be a child 8110 if(name[paramName.length] == '.') { 8111 paramName = name[paramName.length + 1 .. $]; 8112 name = paramName; 8113 int p = 0; 8114 foreach(ch; paramName) { 8115 if(ch == '.' || ch == '[') 8116 break; 8117 p++; 8118 } 8119 8120 // set the child member 8121 switch(paramName) { 8122 foreach(idx, memberName; __traits(allMembers, T)) 8123 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 8124 // data member! 8125 case memberName: 8126 return setVariable(name, paramName, &(__traits(getMember, *what, memberName)), value); 8127 } 8128 default: 8129 // ok, not a member 8130 } 8131 } 8132 } 8133 8134 return false; 8135 } else static if(isSomeString!T || isIntegral!T || isFloatingPoint!T) { 8136 *what = to!T(value); 8137 return true; 8138 } else static if(is(T == bool)) { 8139 *what = value == "1" || value == "yes" || value == "t" || value == "true" || value == "on"; 8140 return true; 8141 } else static if(is(T == K[], K)) { 8142 K tmp; 8143 if(name == paramName) { 8144 // direct - set and append 8145 if(setVariable(name, paramName, &tmp, value)) { 8146 (*what) ~= tmp; 8147 return true; 8148 } else { 8149 return false; 8150 } 8151 } else { 8152 // child, append to last element 8153 // FIXME: what about range violations??? 8154 auto ptr = &(*what)[(*what).length - 1]; 8155 return setVariable(name, paramName, ptr, value); 8156 8157 } 8158 } else static if(is(T == V[K], K, V)) { 8159 // assoc array, name[key] is valid 8160 if(name == paramName) { 8161 // no action necessary 8162 return true; 8163 } else if(name[paramName.length] == '[') { 8164 int count = 1; 8165 auto idx = paramName.length + 1; 8166 while(idx < name.length && count > 0) { 8167 if(name[idx] == '[') 8168 count++; 8169 else if(name[idx] == ']') { 8170 count--; 8171 if(count == 0) break; 8172 } 8173 idx++; 8174 } 8175 if(idx == name.length) 8176 return false; // malformed 8177 8178 auto insideBrackets = name[paramName.length + 1 .. idx]; 8179 auto afterName = name[idx + 1 .. $]; 8180 8181 auto k = to!K(insideBrackets); 8182 V v; 8183 if(auto ptr = k in *what) 8184 v = *ptr; 8185 8186 name = name[0 .. paramName.length]; 8187 //writeln(name, afterName, " ", paramName); 8188 8189 auto ret = setVariable(name ~ afterName, paramName, &v, value); 8190 if(ret) { 8191 (*what)[k] = v; 8192 return true; 8193 } 8194 } 8195 8196 return false; 8197 } else { 8198 static assert(0, "unsupported type for cgi call " ~ T.stringof); 8199 } 8200 8201 //return false; 8202 } 8203 8204 void setArgument(string name, string value) { 8205 int p; 8206 foreach(ch; name) { 8207 if(ch == '.' || ch == '[') 8208 break; 8209 p++; 8210 } 8211 8212 auto paramName = name[0 .. p]; 8213 8214 sw: switch(paramName) { 8215 static if(is(typeof(method) P == __parameters)) 8216 foreach(idx, param; P) { 8217 static if(mustNotBeSetFromWebParams!(P[idx], __traits(getAttributes, P[idx .. idx + 1]))) { 8218 // cannot be set from the outside 8219 } else { 8220 case idents[idx]: 8221 static if(is(param == Cgi.UploadedFile)) { 8222 params[idx] = cgi.files[name]; 8223 } else { 8224 setVariable(name, paramName, ¶ms[idx], value); 8225 } 8226 break sw; 8227 } 8228 } 8229 default: 8230 // ignore; not relevant argument 8231 } 8232 } 8233 8234 if(cgi.requestMethod == Cgi.RequestMethod.GET) { 8235 names = cgi.allGetNamesInOrder; 8236 values = cgi.allGetValuesInOrder; 8237 } else { 8238 names = cgi.allPostNamesInOrder; 8239 values = cgi.allPostValuesInOrder; 8240 } 8241 8242 foreach(idx, name; names) { 8243 setArgument(name, values[idx]); 8244 } 8245 8246 static if(is(ReturnType!method == void)) { 8247 typeof(null) ret; 8248 dg(params); 8249 } else { 8250 auto ret = dg(params); 8251 } 8252 8253 // FIXME: format return values 8254 // options are: json, html, csv. 8255 // also may need to wrap in envelope format: none, html, or json. 8256 return ret; 8257 } 8258 8259 private bool mustNotBeSetFromWebParams(T, attrs...)() { 8260 static if(is(T : const(Cgi))) { 8261 return true; 8262 } else static if(is(T : const(Session!D), D)) { 8263 return true; 8264 } else static if(__traits(compiles, T.getAutomaticallyForCgi(Cgi.init))) { 8265 return true; 8266 } else { 8267 foreach(uda; attrs) 8268 static if(is(uda == ifCalledFromWeb!func, alias func)) 8269 return true; 8270 return false; 8271 } 8272 } 8273 8274 private bool hasIfCalledFromWeb(attrs...)() { 8275 foreach(uda; attrs) 8276 static if(is(uda == ifCalledFromWeb!func, alias func)) 8277 return true; 8278 return false; 8279 } 8280 8281 /++ 8282 Implies POST path for the thing itself, then GET will get the automatic form. 8283 8284 The given customizer, if present, will be called as a filter on the Form object. 8285 8286 History: 8287 Added December 27, 2020 8288 +/ 8289 template AutomaticForm(alias customizer) { } 8290 8291 /+ 8292 Argument conversions: for the most part, it is to!Thing(string). 8293 8294 But arrays and structs are a bit different. Arrays come from the cgi array. Thus 8295 they are passed 8296 8297 arr=foo&arr=bar <-- notice the same name. 8298 8299 Structs are first declared with an empty thing, then have their members set individually, 8300 with dot notation. The members are not required, just the initial declaration. 8301 8302 struct Foo { 8303 int a; 8304 string b; 8305 } 8306 void test(Foo foo){} 8307 8308 foo&foo.a=5&foo.b=str <-- the first foo declares the arg, the others set the members 8309 8310 Arrays of structs use this declaration. 8311 8312 void test(Foo[] foo) {} 8313 8314 foo&foo.a=5&foo.b=bar&foo&foo.a=9 8315 8316 You can use a hidden input field in HTML forms to achieve this. The value of the naked name 8317 declaration is ignored. 8318 8319 Mind that order matters! The declaration MUST come first in the string. 8320 8321 Arrays of struct members follow this rule recursively. 8322 8323 struct Foo { 8324 int[] a; 8325 } 8326 8327 foo&foo.a=1&foo.a=2&foo&foo.a=1 8328 8329 8330 Associative arrays are formatted with brackets, after a declaration, like structs: 8331 8332 foo&foo[key]=value&foo[other_key]=value 8333 8334 8335 Note: for maximum compatibility with outside code, keep your types simple. Some libraries 8336 do not support the strict ordering requirements to work with these struct protocols. 8337 8338 FIXME: also perhaps accept application/json to better work with outside trash. 8339 8340 8341 Return values are also auto-formatted according to user-requested type: 8342 for json, it loops over and converts. 8343 for html, basic types are strings. Arrays are <ol>. Structs are <dl>. Arrays of structs are tables! 8344 +/ 8345 8346 /++ 8347 A web presenter is responsible for rendering things to HTML to be usable 8348 in a web browser. 8349 8350 They are passed as template arguments to the base classes of [WebObject] 8351 8352 Responsible for displaying stuff as HTML. You can put this into your own aggregate 8353 and override it. Use forwarding and specialization to customize it. 8354 8355 When you inherit from it, pass your own class as the CRTP argument. This lets the base 8356 class templates and your overridden templates work with each other. 8357 8358 --- 8359 class MyPresenter : WebPresenter!(MyPresenter) { 8360 @Override 8361 void presentSuccessfulReturnAsHtml(T : CustomType)(Cgi cgi, T ret, typeof(null) meta) { 8362 // present the CustomType 8363 } 8364 @Override 8365 void presentSuccessfulReturnAsHtml(T)(Cgi cgi, T ret, typeof(null) meta) { 8366 // handle everything else via the super class, which will call 8367 // back to your class when appropriate 8368 super.presentSuccessfulReturnAsHtml(cgi, ret); 8369 } 8370 } 8371 --- 8372 8373 The meta argument in there can be overridden by your own facility. 8374 8375 +/ 8376 class WebPresenter(CRTP) { 8377 8378 /// A UDA version of the built-in `override`, to be used for static template polymorphism 8379 /// If you override a plain method, use `override`. If a template, use `@Override`. 8380 enum Override; 8381 8382 string script() { 8383 return ` 8384 `; 8385 } 8386 8387 string style() { 8388 return ` 8389 :root { 8390 --mild-border: #ccc; 8391 --middle-border: #999; 8392 --accent-color: #e8e8e8; 8393 --sidebar-color: #f2f2f2; 8394 } 8395 ` ~ genericFormStyling() ~ genericSiteStyling(); 8396 } 8397 8398 string genericFormStyling() { 8399 return 8400 q"css 8401 table.automatic-data-display { 8402 border-collapse: collapse; 8403 border: solid 1px var(--mild-border); 8404 } 8405 8406 table.automatic-data-display td { 8407 vertical-align: top; 8408 border: solid 1px var(--mild-border); 8409 padding: 2px 4px; 8410 } 8411 8412 table.automatic-data-display th { 8413 border: solid 1px var(--mild-border); 8414 border-bottom: solid 1px var(--middle-border); 8415 padding: 2px 4px; 8416 } 8417 8418 ol.automatic-data-display { 8419 margin: 0px; 8420 list-style-position: inside; 8421 padding: 0px; 8422 } 8423 8424 dl.automatic-data-display { 8425 8426 } 8427 8428 .automatic-form { 8429 max-width: 600px; 8430 } 8431 8432 .form-field { 8433 margin: 0.5em; 8434 padding-left: 0.5em; 8435 } 8436 8437 .label-text { 8438 display: block; 8439 font-weight: bold; 8440 margin-left: -0.5em; 8441 } 8442 8443 .submit-button-holder { 8444 padding-left: 2em; 8445 } 8446 8447 .add-array-button { 8448 8449 } 8450 css"; 8451 } 8452 8453 string genericSiteStyling() { 8454 return 8455 q"css 8456 * { box-sizing: border-box; } 8457 html, body { margin: 0px; } 8458 body { 8459 font-family: sans-serif; 8460 } 8461 header { 8462 background: var(--accent-color); 8463 height: 64px; 8464 } 8465 footer { 8466 background: var(--accent-color); 8467 height: 64px; 8468 } 8469 #site-container { 8470 display: flex; 8471 } 8472 main { 8473 flex: 1 1 auto; 8474 order: 2; 8475 min-height: calc(100vh - 64px - 64px); 8476 padding: 4px; 8477 padding-left: 1em; 8478 } 8479 #sidebar { 8480 flex: 0 0 16em; 8481 order: 1; 8482 background: var(--sidebar-color); 8483 } 8484 css"; 8485 } 8486 8487 import arsd.dom; 8488 Element htmlContainer() { 8489 auto document = new Document(q"html 8490 <!DOCTYPE html> 8491 <html> 8492 <head> 8493 <title>D Application</title> 8494 <link rel="stylesheet" href="style.css" /> 8495 </head> 8496 <body> 8497 <header></header> 8498 <div id="site-container"> 8499 <main></main> 8500 <div id="sidebar"></div> 8501 </div> 8502 <footer></footer> 8503 <script src="script.js"></script> 8504 </body> 8505 </html> 8506 html", true, true); 8507 8508 return document.requireSelector("main"); 8509 } 8510 8511 /// Renders a response as an HTTP error 8512 void renderBasicError(Cgi cgi, int httpErrorCode) { 8513 cgi.setResponseStatus(getHttpCodeText(httpErrorCode)); 8514 auto c = htmlContainer(); 8515 c.innerText = getHttpCodeText(httpErrorCode); 8516 cgi.setResponseContentType("text/html; charset=utf-8"); 8517 cgi.write(c.parentDocument.toString(), true); 8518 } 8519 8520 template methodMeta(alias method) { 8521 enum methodMeta = null; 8522 } 8523 8524 void presentSuccessfulReturn(T, Meta)(Cgi cgi, T ret, Meta meta, string format) { 8525 // FIXME? format? 8526 (cast(CRTP) this).presentSuccessfulReturnAsHtml(cgi, ret, meta); 8527 } 8528 8529 /// typeof(null) (which is also used to represent functions returning `void`) do nothing 8530 /// in the default presenter - allowing the function to have full low-level control over the 8531 /// response. 8532 void presentSuccessfulReturn(T : typeof(null), Meta)(Cgi cgi, T ret, Meta meta, string format) { 8533 // nothing intentionally! 8534 } 8535 8536 /// Redirections are forwarded to [Cgi.setResponseLocation] 8537 void presentSuccessfulReturn(T : Redirection, Meta)(Cgi cgi, T ret, Meta meta, string format) { 8538 cgi.setResponseLocation(ret.to, true, getHttpCodeText(ret.code)); 8539 } 8540 8541 /// Multiple responses deconstruct the algebraic type and forward to the appropriate handler at runtime 8542 void presentSuccessfulReturn(T : MultipleResponses!Types, Types...)(Cgi cgi, T ret, typeof(null) meta, string format) { 8543 bool outputted = false; 8544 foreach(index, type; Types) { 8545 if(ret.contains == index) { 8546 assert(!outputted); 8547 outputted = true; 8548 (cast(CRTP) this).presentSuccessfulReturnAsHtml(cgi, ret.payload[index], meta); 8549 } 8550 } 8551 if(!outputted) 8552 assert(0); 8553 } 8554 8555 /// An instance of the [arsd.dom.FileResource] interface has its own content type; assume it is a download of some sort. 8556 void presentSuccessfulReturn(T : FileResource, Meta)(Cgi cgi, T ret, Meta meta, string format) { 8557 cgi.setCache(true); // not necessarily true but meh 8558 cgi.setResponseContentType(ret.contentType); 8559 cgi.write(ret.getData(), true); 8560 } 8561 8562 /// And the default handler for HTML will call [formatReturnValueAsHtml] and place it inside the [htmlContainer]. 8563 void presentSuccessfulReturnAsHtml(T)(Cgi cgi, T ret, typeof(null) meta) { 8564 auto container = this.htmlContainer(); 8565 container.appendChild(formatReturnValueAsHtml(ret)); 8566 cgi.write(container.parentDocument.toString(), true); 8567 } 8568 8569 /++ 8570 If you override this, you will need to cast the exception type `t` dynamically, 8571 but can then use the template arguments here to refer back to the function. 8572 8573 `func` is an alias to the method itself, and `dg` is a callable delegate to the same 8574 method on the live object. You could, in theory, change arguments and retry, but I 8575 provide that information mostly with the expectation that you will use them to make 8576 useful forms or richer error messages for the user. 8577 +/ 8578 void presentExceptionAsHtml(alias func, T)(Cgi cgi, Throwable t, T dg) { 8579 presentExceptionAsHtmlImpl(cgi, t, createAutomaticFormForFunction!(func)(dg)); 8580 } 8581 8582 void presentExceptionAsHtmlImpl(Cgi cgi, Throwable t, Form automaticForm) { 8583 if(auto mae = cast(MissingArgumentException) t) { 8584 auto container = this.htmlContainer(); 8585 if(cgi.requestMethod == Cgi.RequestMethod.POST) 8586 container.appendChild(Element.make("p", "Argument `" ~ mae.argumentName ~ "` of type `" ~ mae.argumentType ~ "` is missing")); 8587 container.appendChild(automaticForm); 8588 8589 cgi.write(container.parentDocument.toString(), true); 8590 } else { 8591 auto container = this.htmlContainer(); 8592 8593 // import std.stdio; writeln(t.toString()); 8594 8595 container.appendChild(exceptionToElement(t)); 8596 8597 container.addChild("h4", "GET"); 8598 foreach(k, v; cgi.get) { 8599 auto deets = container.addChild("details"); 8600 deets.addChild("summary", k); 8601 deets.addChild("div", v); 8602 } 8603 8604 container.addChild("h4", "POST"); 8605 foreach(k, v; cgi.post) { 8606 auto deets = container.addChild("details"); 8607 deets.addChild("summary", k); 8608 deets.addChild("div", v); 8609 } 8610 8611 8612 if(!cgi.outputtedResponseData) 8613 cgi.setResponseStatus("500 Internal Server Error"); 8614 cgi.write(container.parentDocument.toString(), true); 8615 } 8616 } 8617 8618 Element exceptionToElement(Throwable t) { 8619 auto div = Element.make("div"); 8620 div.addClass("exception-display"); 8621 8622 div.addChild("p", t.msg); 8623 div.addChild("p", "Inner code origin: " ~ typeid(t).name ~ "@" ~ t.file ~ ":" ~ to!string(t.line)); 8624 8625 auto pre = div.addChild("pre"); 8626 string s; 8627 s = t.toString(); 8628 Element currentBox; 8629 bool on = false; 8630 foreach(line; s.splitLines) { 8631 if(!on && line.startsWith("-----")) 8632 on = true; 8633 if(!on) continue; 8634 if(line.indexOf("arsd/") != -1) { 8635 if(currentBox is null) { 8636 currentBox = pre.addChild("details"); 8637 currentBox.addChild("summary", "Framework code"); 8638 } 8639 currentBox.addChild("span", line ~ "\n"); 8640 } else { 8641 pre.addChild("span", line ~ "\n"); 8642 currentBox = null; 8643 } 8644 } 8645 8646 return div; 8647 } 8648 8649 /++ 8650 Returns an element for a particular type 8651 +/ 8652 Element elementFor(T)(string displayName, string name) { 8653 import std.traits; 8654 8655 auto div = Element.make("div"); 8656 div.addClass("form-field"); 8657 8658 static if(is(T == struct)) { 8659 if(displayName !is null) 8660 div.addChild("span", displayName, "label-text"); 8661 auto fieldset = div.addChild("fieldset"); 8662 fieldset.addChild("legend", beautify(T.stringof)); // FIXME 8663 fieldset.addChild("input", name); 8664 foreach(idx, memberName; __traits(allMembers, T)) 8665 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 8666 fieldset.appendChild(elementFor!(typeof(__traits(getMember, T, memberName)))(beautify(memberName), name ~ "." ~ memberName)); 8667 } 8668 } else static if(isSomeString!T || isIntegral!T || isFloatingPoint!T) { 8669 Element lbl; 8670 if(displayName !is null) { 8671 lbl = div.addChild("label"); 8672 lbl.addChild("span", displayName, "label-text"); 8673 lbl.appendText(" "); 8674 } else { 8675 lbl = div; 8676 } 8677 auto i = lbl.addChild("input", name); 8678 i.attrs.name = name; 8679 static if(isSomeString!T) 8680 i.attrs.type = "text"; 8681 else 8682 i.attrs.type = "number"; 8683 i.attrs.value = to!string(T.init); 8684 } else static if(is(T == bool)) { 8685 Element lbl; 8686 if(displayName !is null) { 8687 lbl = div.addChild("label"); 8688 lbl.addChild("span", displayName, "label-text"); 8689 lbl.appendText(" "); 8690 } else { 8691 lbl = div; 8692 } 8693 auto i = lbl.addChild("input", name); 8694 i.attrs.type = "checkbox"; 8695 i.attrs.value = "true"; 8696 i.attrs.name = name; 8697 } else static if(is(T == Cgi.UploadedFile)) { 8698 Element lbl; 8699 if(displayName !is null) { 8700 lbl = div.addChild("label"); 8701 lbl.addChild("span", displayName, "label-text"); 8702 lbl.appendText(" "); 8703 } else { 8704 lbl = div; 8705 } 8706 auto i = lbl.addChild("input", name); 8707 i.attrs.name = name; 8708 i.attrs.type = "file"; 8709 } else static if(is(T == K[], K)) { 8710 auto templ = div.addChild("template"); 8711 templ.appendChild(elementFor!(K)(null, name)); 8712 if(displayName !is null) 8713 div.addChild("span", displayName, "label-text"); 8714 auto btn = div.addChild("button"); 8715 btn.addClass("add-array-button"); 8716 btn.attrs.type = "button"; 8717 btn.innerText = "Add"; 8718 btn.attrs.onclick = q{ 8719 var a = document.importNode(this.parentNode.firstChild.content, true); 8720 this.parentNode.insertBefore(a, this); 8721 }; 8722 } else static if(is(T == V[K], K, V)) { 8723 div.innerText = "assoc array not implemented for automatic form at this time"; 8724 } else { 8725 static assert(0, "unsupported type for cgi call " ~ T.stringof); 8726 } 8727 8728 8729 return div; 8730 } 8731 8732 /// creates a form for gathering the function's arguments 8733 Form createAutomaticFormForFunction(alias method, T)(T dg) { 8734 8735 auto form = cast(Form) Element.make("form"); 8736 8737 form.method = "POST"; // FIXME 8738 8739 form.addClass("automatic-form"); 8740 8741 string formDisplayName = beautify(__traits(identifier, method)); 8742 foreach(attr; __traits(getAttributes, method)) 8743 static if(is(typeof(attr) == DisplayName)) 8744 formDisplayName = attr.name; 8745 form.addChild("h3", formDisplayName); 8746 8747 import std.traits; 8748 8749 //Parameters!method params; 8750 //alias idents = ParameterIdentifierTuple!method; 8751 //alias defaults = ParameterDefaults!method; 8752 8753 static if(is(typeof(method) P == __parameters)) 8754 foreach(idx, _; P) {{ 8755 8756 alias param = P[idx .. idx + 1]; 8757 8758 static if(!mustNotBeSetFromWebParams!(param[0], __traits(getAttributes, param))) { 8759 string displayName = beautify(__traits(identifier, param)); 8760 foreach(attr; __traits(getAttributes, param)) 8761 static if(is(typeof(attr) == DisplayName)) 8762 displayName = attr.name; 8763 auto i = form.appendChild(elementFor!(param)(displayName, __traits(identifier, param))); 8764 if(i.querySelector("input[type=file]") !is null) 8765 form.setAttribute("enctype", "multipart/form-data"); 8766 } 8767 }} 8768 8769 form.addChild("div", Html(`<input type="submit" value="Submit" />`), "submit-button-holder"); 8770 8771 return form; 8772 } 8773 8774 /// creates a form for gathering object members (for the REST object thing right now) 8775 Form createAutomaticFormForObject(T)(T obj) { 8776 auto form = cast(Form) Element.make("form"); 8777 8778 form.addClass("automatic-form"); 8779 8780 form.addChild("h3", beautify(__traits(identifier, T))); 8781 8782 import std.traits; 8783 8784 //Parameters!method params; 8785 //alias idents = ParameterIdentifierTuple!method; 8786 //alias defaults = ParameterDefaults!method; 8787 8788 foreach(idx, memberName; __traits(derivedMembers, T)) {{ 8789 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 8790 string displayName = beautify(memberName); 8791 foreach(attr; __traits(getAttributes, __traits(getMember, T, memberName))) 8792 static if(is(typeof(attr) == DisplayName)) 8793 displayName = attr.name; 8794 form.appendChild(elementFor!(typeof(__traits(getMember, T, memberName)))(displayName, memberName)); 8795 8796 form.setValue(memberName, to!string(__traits(getMember, obj, memberName))); 8797 }}} 8798 8799 form.addChild("div", Html(`<input type="submit" value="Submit" />`), "submit-button-holder"); 8800 8801 return form; 8802 } 8803 8804 /// 8805 Element formatReturnValueAsHtml(T)(T t) { 8806 import std.traits; 8807 8808 static if(is(T == typeof(null))) { 8809 return Element.make("span"); 8810 } else static if(is(T : Element)) { 8811 return t; 8812 } else static if(is(T == MultipleResponses!Types, Types...)) { 8813 foreach(index, type; Types) { 8814 if(t.contains == index) 8815 return formatReturnValueAsHtml(t.payload[index]); 8816 } 8817 assert(0); 8818 } else static if(is(T == Paginated!E, E)) { 8819 auto e = Element.make("div").addClass("paginated-result"); 8820 e.appendChild(formatReturnValueAsHtml(t.items)); 8821 if(t.nextPageUrl.length) 8822 e.appendChild(Element.make("a", "Next Page", t.nextPageUrl)); 8823 return e; 8824 } else static if(isIntegral!T || isSomeString!T || isFloatingPoint!T) { 8825 return Element.make("span", to!string(t), "automatic-data-display"); 8826 } else static if(is(T == V[K], K, V)) { 8827 auto dl = Element.make("dl"); 8828 dl.addClass("automatic-data-display associative-array"); 8829 foreach(k, v; t) { 8830 dl.addChild("dt", to!string(k)); 8831 dl.addChild("dd", formatReturnValueAsHtml(v)); 8832 } 8833 return dl; 8834 } else static if(is(T == struct)) { 8835 auto dl = Element.make("dl"); 8836 dl.addClass("automatic-data-display struct"); 8837 8838 foreach(idx, memberName; __traits(allMembers, T)) 8839 static if(__traits(compiles, __traits(getMember, T, memberName).offsetof)) { 8840 dl.addChild("dt", beautify(memberName)); 8841 dl.addChild("dd", formatReturnValueAsHtml(__traits(getMember, t, memberName))); 8842 } 8843 8844 return dl; 8845 } else static if(is(T == bool)) { 8846 return Element.make("span", t ? "true" : "false", "automatic-data-display"); 8847 } else static if(is(T == E[], E)) { 8848 static if(is(E : RestObject!Proxy, Proxy)) { 8849 // treat RestObject similar to struct 8850 auto table = cast(Table) Element.make("table"); 8851 table.addClass("automatic-data-display"); 8852 string[] names; 8853 foreach(idx, memberName; __traits(derivedMembers, E)) 8854 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 8855 names ~= beautify(memberName); 8856 } 8857 table.appendHeaderRow(names); 8858 8859 foreach(l; t) { 8860 auto tr = table.appendRow(); 8861 foreach(idx, memberName; __traits(derivedMembers, E)) 8862 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 8863 static if(memberName == "id") { 8864 string val = to!string(__traits(getMember, l, memberName)); 8865 tr.addChild("td", Element.make("a", val, E.stringof.toLower ~ "s/" ~ val)); // FIXME 8866 } else { 8867 tr.addChild("td", formatReturnValueAsHtml(__traits(getMember, l, memberName))); 8868 } 8869 } 8870 } 8871 8872 return table; 8873 } else static if(is(E == struct)) { 8874 // an array of structs is kinda special in that I like 8875 // having those formatted as tables. 8876 auto table = cast(Table) Element.make("table"); 8877 table.addClass("automatic-data-display"); 8878 string[] names; 8879 foreach(idx, memberName; __traits(allMembers, E)) 8880 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 8881 names ~= beautify(memberName); 8882 } 8883 table.appendHeaderRow(names); 8884 8885 foreach(l; t) { 8886 auto tr = table.appendRow(); 8887 foreach(idx, memberName; __traits(allMembers, E)) 8888 static if(__traits(compiles, __traits(getMember, E, memberName).offsetof)) { 8889 tr.addChild("td", formatReturnValueAsHtml(__traits(getMember, l, memberName))); 8890 } 8891 } 8892 8893 return table; 8894 } else { 8895 // otherwise, I will just make a list. 8896 auto ol = Element.make("ol"); 8897 ol.addClass("automatic-data-display"); 8898 foreach(e; t) 8899 ol.addChild("li", formatReturnValueAsHtml(e)); 8900 return ol; 8901 } 8902 } else static assert(0, "bad return value for cgi call " ~ T.stringof); 8903 8904 assert(0); 8905 } 8906 8907 } 8908 8909 /++ 8910 The base class for the [dispatcher] function and object support. 8911 +/ 8912 class WebObject { 8913 //protected Cgi cgi; 8914 8915 protected void initialize(Cgi cgi) { 8916 //this.cgi = cgi; 8917 } 8918 } 8919 8920 /++ 8921 Can return one of the given types, decided at runtime. The syntax 8922 is to declare all the possible types in the return value, then you 8923 can `return typeof(return)(...value...)` to construct it. 8924 8925 It has an auto-generated constructor for each value it can hold. 8926 8927 --- 8928 MultipleResponses!(Redirection, string) getData(int how) { 8929 if(how & 1) 8930 return typeof(return)(Redirection("http://dpldocs.info/")); 8931 else 8932 return typeof(return)("hi there!"); 8933 } 8934 --- 8935 8936 If you have lots of returns, you could, inside the function, `alias r = typeof(return);` to shorten it a little. 8937 +/ 8938 struct MultipleResponses(T...) { 8939 private size_t contains; 8940 private union { 8941 private T payload; 8942 } 8943 8944 static foreach(index, type; T) 8945 public this(type t) { 8946 contains = index; 8947 payload[index] = t; 8948 } 8949 8950 /++ 8951 This is primarily for testing. It is your way of getting to the response. 8952 8953 Let's say you wanted to test that one holding a Redirection and a string actually 8954 holds a string, by name of "test": 8955 8956 --- 8957 auto valueToTest = your_test_function(); 8958 8959 valueToTest.visit!( 8960 (Redirection) { assert(0); }, // got a redirection instead of a string, fail the test 8961 (string s) { assert(s == "test"); } // right value, go ahead and test it. 8962 ); 8963 --- 8964 +/ 8965 void visit(Handlers...)() { 8966 template findHandler(type, HandlersToCheck...) { 8967 static if(HandlersToCheck.length == 0) 8968 alias findHandler = void; 8969 else { 8970 static if(is(typeof(HandlersToCheck[0](type.init)))) 8971 alias findHandler = handler; 8972 else 8973 alias findHandler = findHandler!(type, HandlersToCheck[1 .. $]); 8974 } 8975 } 8976 foreach(index, type; T) { 8977 alias handler = findHandler!(type, Handlers); 8978 static if(is(handler == void)) 8979 static assert(0, "Type " ~ type.stringof ~ " was not handled by visitor"); 8980 else { 8981 if(index == contains) 8982 handler(payload[index]); 8983 } 8984 } 8985 } 8986 8987 /+ 8988 auto toArsdJsvar()() { 8989 import arsd.jsvar; 8990 return var(null); 8991 } 8992 +/ 8993 } 8994 8995 struct RawResponse { 8996 int code; 8997 string[] headers; 8998 const(ubyte)[] responseBody; 8999 } 9000 9001 /++ 9002 You can return this from [WebObject] subclasses for redirections. 9003 9004 (though note the static types means that class must ALWAYS redirect if 9005 you return this directly. You might want to return [MultipleResponses] if it 9006 can be conditional) 9007 +/ 9008 struct Redirection { 9009 string to; /// The URL to redirect to. 9010 int code = 303; /// The HTTP code to retrn. 9011 } 9012 9013 /++ 9014 Serves a class' methods, as a kind of low-state RPC over the web. To be used with [dispatcher]. 9015 9016 Usage of this function will add a dependency on [arsd.dom] and [arsd.jsvar] unless you have overriden 9017 the presenter in the dispatcher. 9018 9019 FIXME: explain this better 9020 9021 You can overload functions to a limited extent: you can provide a zero-arg and non-zero-arg function, 9022 and non-zero-arg functions can filter via UDAs for various http methods. Do not attempt other overloads, 9023 the runtime result of that is undefined. 9024 9025 A method is assumed to allow any http method unless it lists some in UDAs, in which case it is limited to only those. 9026 (this might change, like maybe i will use pure as an indicator GET is ok. idk.) 9027 9028 $(WARNING 9029 --- 9030 // legal in D, undefined runtime behavior with cgi.d, it may call either method 9031 // even if you put different URL udas on it, the current code ignores them. 9032 void foo(int a) {} 9033 void foo(string a) {} 9034 --- 9035 ) 9036 9037 See_Also: [serveRestObject], [serveStaticFile] 9038 +/ 9039 auto serveApi(T)(string urlPrefix) { 9040 assert(urlPrefix[$ - 1] == '/'); 9041 return serveApiInternal!T(urlPrefix); 9042 } 9043 9044 private string nextPieceFromSlash(ref string remainingUrl) { 9045 if(remainingUrl.length == 0) 9046 return remainingUrl; 9047 int slash = 0; 9048 while(slash < remainingUrl.length && remainingUrl[slash] != '/') // && remainingUrl[slash] != '.') 9049 slash++; 9050 9051 // I am specifically passing `null` to differentiate it vs empty string 9052 // so in your ctor, `items` means new T(null) and `items/` means new T("") 9053 auto ident = remainingUrl.length == 0 ? null : remainingUrl[0 .. slash]; 9054 // so if it is the last item, the dot can be used to load an alternative view 9055 // otherwise tho the dot is considered part of the identifier 9056 // FIXME 9057 9058 // again notice "" vs null here! 9059 if(slash == remainingUrl.length) 9060 remainingUrl = null; 9061 else 9062 remainingUrl = remainingUrl[slash + 1 .. $]; 9063 9064 return ident; 9065 } 9066 9067 enum AddTrailingSlash; 9068 9069 private auto serveApiInternal(T)(string urlPrefix) { 9070 9071 import arsd.dom; 9072 import arsd.jsvar; 9073 9074 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, immutable void* details) { 9075 string remainingUrl = cgi.pathInfo[urlPrefix.length .. $]; 9076 9077 try { 9078 // see duplicated code below by searching subresource_ctor 9079 // also see mustNotBeSetFromWebParams 9080 9081 static if(is(typeof(T.__ctor) P == __parameters)) { 9082 P params; 9083 9084 foreach(pidx, param; P) { 9085 static if(is(param : Cgi)) { 9086 static assert(!is(param == immutable)); 9087 cast() params[pidx] = cgi; 9088 } else static if(is(param == Session!D, D)) { 9089 static assert(!is(param == immutable)); 9090 cast() params[pidx] = cgi.getSessionObject!D(); 9091 9092 } else { 9093 static if(hasIfCalledFromWeb!(__traits(getAttributes, P[pidx .. pidx + 1]))) { 9094 foreach(uda; __traits(getAttributes, P[pidx .. pidx + 1])) { 9095 static if(is(uda == ifCalledFromWeb!func, alias func)) { 9096 static if(is(typeof(func(cgi)))) 9097 params[pidx] = func(cgi); 9098 else 9099 params[pidx] = func(); 9100 } 9101 } 9102 } else { 9103 9104 static if(__traits(compiles, { params[pidx] = param.getAutomaticallyForCgi(cgi); } )) { 9105 params[pidx] = param.getAutomaticallyForCgi(cgi); 9106 } else static if(is(param == string)) { 9107 auto ident = nextPieceFromSlash(remainingUrl); 9108 params[pidx] = ident; 9109 } else static assert(0, "illegal type for subresource " ~ param.stringof); 9110 } 9111 } 9112 } 9113 9114 auto obj = new T(params); 9115 } else { 9116 auto obj = new T(); 9117 } 9118 9119 return internalHandlerWithObject(obj, remainingUrl, cgi, presenter); 9120 } catch(Throwable t) { 9121 switch(cgi.request("format", "html")) { 9122 case "html": 9123 static void dummy() {} 9124 presenter.presentExceptionAsHtml!(dummy)(cgi, t, &dummy); 9125 return true; 9126 case "json": 9127 var envelope = var.emptyObject; 9128 envelope.success = false; 9129 envelope.result = null; 9130 envelope.error = t.toString(); 9131 cgi.setResponseContentType("application/json"); 9132 cgi.write(envelope.toJson(), true); 9133 return true; 9134 default: 9135 throw t; 9136 // return true; 9137 } 9138 // return true; 9139 } 9140 9141 assert(0); 9142 } 9143 9144 static bool internalHandlerWithObject(T, Presenter)(T obj, string remainingUrl, Cgi cgi, Presenter presenter) { 9145 9146 obj.initialize(cgi); 9147 9148 /+ 9149 Overload rules: 9150 Any unique combination of HTTP verb and url path can be dispatched to function overloads 9151 statically. 9152 9153 Moreover, some args vs no args can be overloaded dynamically. 9154 +/ 9155 9156 auto methodNameFromUrl = nextPieceFromSlash(remainingUrl); 9157 /+ 9158 auto orig = remainingUrl; 9159 assert(0, 9160 (orig is null ? "__null" : orig) 9161 ~ " .. " ~ 9162 (methodNameFromUrl is null ? "__null" : methodNameFromUrl)); 9163 +/ 9164 9165 if(methodNameFromUrl is null) 9166 methodNameFromUrl = "__null"; 9167 9168 string hack = to!string(cgi.requestMethod) ~ " " ~ methodNameFromUrl; 9169 9170 if(remainingUrl.length) 9171 hack ~= "/"; 9172 9173 switch(hack) { 9174 foreach(methodName; __traits(derivedMembers, T)) 9175 static if(methodName != "__ctor") 9176 foreach(idx, overload; __traits(getOverloads, T, methodName)) { 9177 static if(is(typeof(overload) P == __parameters)) 9178 static if(is(typeof(overload) R == return)) 9179 static if(__traits(getProtection, overload) == "public" || __traits(getProtection, overload) == "export") 9180 { 9181 static foreach(urlNameForMethod; urlNamesForMethod!(overload, urlify(methodName))) 9182 case urlNameForMethod: 9183 9184 static if(is(R : WebObject)) { 9185 // if it returns a WebObject, it is considered a subresource. That means the url is dispatched like the ctor above. 9186 9187 // the only argument it is allowed to take, outside of cgi, session, and set up thingies, is a single string 9188 9189 // subresource_ctor 9190 // also see mustNotBeSetFromWebParams 9191 9192 P params; 9193 9194 string ident; 9195 9196 foreach(pidx, param; P) { 9197 static if(is(param : Cgi)) { 9198 static assert(!is(param == immutable)); 9199 cast() params[pidx] = cgi; 9200 } else static if(is(param == typeof(presenter))) { 9201 cast() param[pidx] = presenter; 9202 } else static if(is(param == Session!D, D)) { 9203 static assert(!is(param == immutable)); 9204 cast() params[pidx] = cgi.getSessionObject!D(); 9205 } else { 9206 static if(hasIfCalledFromWeb!(__traits(getAttributes, P[pidx .. pidx + 1]))) { 9207 foreach(uda; __traits(getAttributes, P[pidx .. pidx + 1])) { 9208 static if(is(uda == ifCalledFromWeb!func, alias func)) { 9209 static if(is(typeof(func(cgi)))) 9210 params[pidx] = func(cgi); 9211 else 9212 params[pidx] = func(); 9213 } 9214 } 9215 } else { 9216 9217 static if(__traits(compiles, { params[pidx] = param.getAutomaticallyForCgi(cgi); } )) { 9218 params[pidx] = param.getAutomaticallyForCgi(cgi); 9219 } else static if(is(param == string)) { 9220 ident = nextPieceFromSlash(remainingUrl); 9221 if(ident is null) { 9222 // trailing slash mandated on subresources 9223 cgi.setResponseLocation(cgi.pathInfo ~ "/"); 9224 return true; 9225 } else { 9226 params[pidx] = ident; 9227 } 9228 } else static assert(0, "illegal type for subresource " ~ param.stringof); 9229 } 9230 } 9231 } 9232 9233 auto nobj = (__traits(getOverloads, obj, methodName)[idx])(ident); 9234 return internalHandlerWithObject!(typeof(nobj), Presenter)(nobj, remainingUrl, cgi, presenter); 9235 } else { 9236 // 404 it if any url left - not a subresource means we don't get to play with that! 9237 if(remainingUrl.length) 9238 return false; 9239 9240 bool automaticForm; 9241 9242 foreach(attr; __traits(getAttributes, overload)) 9243 static if(is(attr == AddTrailingSlash)) { 9244 if(remainingUrl is null) { 9245 cgi.setResponseLocation(cgi.pathInfo ~ "/"); 9246 return true; 9247 } 9248 } else static if(__traits(isSame, AutomaticForm, attr)) { 9249 automaticForm = true; 9250 } 9251 9252 /+ 9253 int zeroArgOverload = -1; 9254 int overloadCount = cast(int) __traits(getOverloads, T, methodName).length; 9255 bool calledWithZeroArgs = true; 9256 foreach(k, v; cgi.get) 9257 if(k != "format") { 9258 calledWithZeroArgs = false; 9259 break; 9260 } 9261 foreach(k, v; cgi.post) 9262 if(k != "format") { 9263 calledWithZeroArgs = false; 9264 break; 9265 } 9266 9267 // first, we need to go through and see if there is an empty one, since that 9268 // changes inside. But otherwise, all the stuff I care about can be done via 9269 // simple looping (other improper overloads might be flagged for runtime semantic check) 9270 // 9271 // an argument of type Cgi is ignored for these purposes 9272 static foreach(idx, overload; __traits(getOverloads, T, methodName)) {{ 9273 static if(is(typeof(overload) P == __parameters)) 9274 static if(P.length == 0) 9275 zeroArgOverload = cast(int) idx; 9276 else static if(P.length == 1 && is(P[0] : Cgi)) 9277 zeroArgOverload = cast(int) idx; 9278 }} 9279 // FIXME: static assert if there are multiple non-zero-arg overloads usable with a single http method. 9280 bool overloadHasBeenCalled = false; 9281 static foreach(idx, overload; __traits(getOverloads, T, methodName)) {{ 9282 bool callFunction = true; 9283 // there is a zero arg overload and this is NOT it, and we have zero args - don't call this 9284 if(overloadCount > 1 && zeroArgOverload != -1 && idx != zeroArgOverload && calledWithZeroArgs) 9285 callFunction = false; 9286 // if this is the zero-arg overload, obviously it cannot be called if we got any args. 9287 if(overloadCount > 1 && idx == zeroArgOverload && !calledWithZeroArgs) 9288 callFunction = false; 9289 9290 // FIXME: so if you just add ?foo it will give the error below even when. this might not be a great idea. 9291 9292 bool hadAnyMethodRestrictions = false; 9293 bool foundAcceptableMethod = false; 9294 foreach(attr; __traits(getAttributes, overload)) { 9295 static if(is(typeof(attr) == Cgi.RequestMethod)) { 9296 hadAnyMethodRestrictions = true; 9297 if(attr == cgi.requestMethod) 9298 foundAcceptableMethod = true; 9299 } 9300 } 9301 9302 if(hadAnyMethodRestrictions && !foundAcceptableMethod) 9303 callFunction = false; 9304 9305 /+ 9306 The overloads we really want to allow are the sane ones 9307 from the web perspective. Which is likely on HTTP verbs, 9308 for the most part, but might also be potentially based on 9309 some args vs zero args, or on argument names. Can't really 9310 do argument types very reliable through the web though; those 9311 should probably be different URLs. 9312 9313 Even names I feel is better done inside the function, so I'm not 9314 going to support that here. But the HTTP verbs and zero vs some 9315 args makes sense - it lets you define custom forms pretty easily. 9316 9317 Moreover, I'm of the opinion that empty overload really only makes 9318 sense on GET for this case. On a POST, it is just a missing argument 9319 exception and that should be handled by the presenter. But meh, I'll 9320 let the user define that, D only allows one empty arg thing anyway 9321 so the method UDAs are irrelevant. 9322 +/ 9323 if(callFunction) 9324 +/ 9325 9326 if(automaticForm && cgi.requestMethod == Cgi.RequestMethod.GET) { 9327 // Should I still show the form on a json thing? idk... 9328 auto ret = presenter.createAutomaticFormForFunction!((__traits(getOverloads, obj, methodName)[idx]))(&(__traits(getOverloads, obj, methodName)[idx])); 9329 presenter.presentSuccessfulReturn(cgi, ret, presenter.methodMeta!(__traits(getOverloads, obj, methodName)[idx]), "html"); 9330 return true; 9331 } 9332 switch(cgi.request("format", defaultFormat!overload())) { 9333 case "html": 9334 // a void return (or typeof(null) lol) means you, the user, is doing it yourself. Gives full control. 9335 try { 9336 9337 auto ret = callFromCgi!(__traits(getOverloads, obj, methodName)[idx])(&(__traits(getOverloads, obj, methodName)[idx]), cgi); 9338 presenter.presentSuccessfulReturn(cgi, ret, presenter.methodMeta!(__traits(getOverloads, obj, methodName)[idx]), "html"); 9339 } catch(Throwable t) { 9340 presenter.presentExceptionAsHtml!(__traits(getOverloads, obj, methodName)[idx])(cgi, t, &(__traits(getOverloads, obj, methodName)[idx])); 9341 } 9342 return true; 9343 case "json": 9344 auto ret = callFromCgi!(__traits(getOverloads, obj, methodName)[idx])(&(__traits(getOverloads, obj, methodName)[idx]), cgi); 9345 static if(is(typeof(ret) == MultipleResponses!Types, Types...)) { 9346 var json; 9347 foreach(index, type; Types) { 9348 if(ret.contains == index) 9349 json = ret.payload[index]; 9350 } 9351 } else { 9352 var json = ret; 9353 } 9354 var envelope = json; // var.emptyObject; 9355 /* 9356 envelope.success = true; 9357 envelope.result = json; 9358 envelope.error = null; 9359 */ 9360 cgi.setResponseContentType("application/json"); 9361 cgi.write(envelope.toJson(), true); 9362 return true; 9363 default: 9364 cgi.setResponseStatus("406 Not Acceptable"); // not exactly but sort of. 9365 return true; 9366 } 9367 //}} 9368 9369 //cgi.header("Accept: POST"); // FIXME list the real thing 9370 //cgi.setResponseStatus("405 Method Not Allowed"); // again, not exactly, but sort of. no overload matched our args, almost certainly due to http verb filtering. 9371 //return true; 9372 } 9373 } 9374 } 9375 case "GET script.js": 9376 cgi.setResponseContentType("text/javascript"); 9377 cgi.gzipResponse = true; 9378 cgi.write(presenter.script(), true); 9379 return true; 9380 case "GET style.css": 9381 cgi.setResponseContentType("text/css"); 9382 cgi.gzipResponse = true; 9383 cgi.write(presenter.style(), true); 9384 return true; 9385 default: 9386 return false; 9387 } 9388 9389 assert(0); 9390 } 9391 return DispatcherDefinition!internalHandler(urlPrefix, false); 9392 } 9393 9394 string defaultFormat(alias method)() { 9395 bool nonConstConditionForWorkingAroundASpuriousDmdWarning = true; 9396 foreach(attr; __traits(getAttributes, method)) { 9397 static if(is(typeof(attr) == DefaultFormat)) { 9398 if(nonConstConditionForWorkingAroundASpuriousDmdWarning) 9399 return attr.value; 9400 } 9401 } 9402 return "html"; 9403 } 9404 9405 struct Paginated(T) { 9406 T[] items; 9407 string nextPageUrl; 9408 } 9409 9410 template urlNamesForMethod(alias method, string default_) { 9411 string[] helper() { 9412 auto verb = Cgi.RequestMethod.GET; 9413 bool foundVerb = false; 9414 bool foundNoun = false; 9415 9416 string def = default_; 9417 9418 bool hasAutomaticForm = false; 9419 9420 foreach(attr; __traits(getAttributes, method)) { 9421 static if(is(typeof(attr) == Cgi.RequestMethod)) { 9422 verb = attr; 9423 if(foundVerb) 9424 assert(0, "Multiple http verbs on one function is not currently supported"); 9425 foundVerb = true; 9426 } 9427 static if(is(typeof(attr) == UrlName)) { 9428 if(foundNoun) 9429 assert(0, "Multiple url names on one function is not currently supported"); 9430 foundNoun = true; 9431 def = attr.name; 9432 } 9433 static if(__traits(isSame, attr, AutomaticForm)) { 9434 hasAutomaticForm = true; 9435 } 9436 } 9437 9438 if(def is null) 9439 def = "__null"; 9440 9441 string[] ret; 9442 9443 static if(is(typeof(method) R == return)) { 9444 static if(is(R : WebObject)) { 9445 def ~= "/"; 9446 foreach(v; __traits(allMembers, Cgi.RequestMethod)) 9447 ret ~= v ~ " " ~ def; 9448 } else { 9449 if(hasAutomaticForm) { 9450 ret ~= "GET " ~ def; 9451 ret ~= "POST " ~ def; 9452 } else { 9453 ret ~= to!string(verb) ~ " " ~ def; 9454 } 9455 } 9456 } else static assert(0); 9457 9458 return ret; 9459 } 9460 enum urlNamesForMethod = helper(); 9461 } 9462 9463 9464 enum AccessCheck { 9465 allowed, 9466 denied, 9467 nonExistant, 9468 } 9469 9470 enum Operation { 9471 show, 9472 create, 9473 replace, 9474 remove, 9475 update 9476 } 9477 9478 enum UpdateResult { 9479 accessDenied, 9480 noSuchResource, 9481 success, 9482 failure, 9483 unnecessary 9484 } 9485 9486 enum ValidationResult { 9487 valid, 9488 invalid 9489 } 9490 9491 9492 /++ 9493 The base of all REST objects, to be used with [serveRestObject] and [serveRestCollectionOf]. 9494 +/ 9495 class RestObject(CRTP) : WebObject { 9496 9497 import arsd.dom; 9498 import arsd.jsvar; 9499 9500 /// Prepare the object to be shown. 9501 void show() {} 9502 /// ditto 9503 void show(string urlId) { 9504 load(urlId); 9505 show(); 9506 } 9507 9508 ValidationResult delegate(typeof(this)) validateFromReflection; 9509 Element delegate(typeof(this)) toHtmlFromReflection; 9510 var delegate(typeof(this)) toJsonFromReflection; 9511 9512 /// Override this to provide access control to this object. 9513 AccessCheck accessCheck(string urlId, Operation operation) { 9514 return AccessCheck.allowed; 9515 } 9516 9517 ValidationResult validate() { 9518 if(validateFromReflection !is null) 9519 return validateFromReflection(this); 9520 return ValidationResult.valid; 9521 } 9522 9523 // The functions with more arguments are the low-level ones, 9524 // they forward to the ones with fewer arguments by default. 9525 9526 // POST on a parent collection - this is called from a collection class after the members are updated 9527 /++ 9528 Given a populated object, this creates a new entry. Returns the url identifier 9529 of the new object. 9530 +/ 9531 string create(scope void delegate() applyChanges) { 9532 return null; 9533 } 9534 9535 void replace() { 9536 save(); 9537 } 9538 void replace(string urlId, scope void delegate() applyChanges) { 9539 load(urlId); 9540 applyChanges(); 9541 replace(); 9542 } 9543 9544 void update(string[] fieldList) { 9545 save(); 9546 } 9547 void update(string urlId, scope void delegate() applyChanges, string[] fieldList) { 9548 load(urlId); 9549 applyChanges(); 9550 update(fieldList); 9551 } 9552 9553 void remove() {} 9554 9555 void remove(string urlId) { 9556 load(urlId); 9557 remove(); 9558 } 9559 9560 abstract void load(string urlId); 9561 abstract void save(); 9562 9563 Element toHtml() { 9564 if(toHtmlFromReflection) 9565 return toHtmlFromReflection(this); 9566 else 9567 assert(0); 9568 } 9569 9570 var toJson() { 9571 if(toJsonFromReflection) 9572 return toJsonFromReflection(this); 9573 else 9574 assert(0); 9575 } 9576 9577 /+ 9578 auto structOf(this This) { 9579 9580 } 9581 +/ 9582 } 9583 9584 // FIXME XSRF token, prolly can just put in a cookie and then it needs to be copied to header or form hidden value 9585 // https://use-the-index-luke.com/sql/partial-results/fetch-next-page 9586 9587 /++ 9588 Base class for REST collections. 9589 +/ 9590 class CollectionOf(Obj) : RestObject!(CollectionOf) { 9591 /// You might subclass this and use the cgi object's query params 9592 /// to implement a search filter, for example. 9593 /// 9594 /// FIXME: design a way to auto-generate that form 9595 /// (other than using the WebObject thing above lol 9596 // it'll prolly just be some searchParams UDA or maybe an enum. 9597 // 9598 // pagination too perhaps. 9599 // 9600 // and sorting too 9601 IndexResult index() { return IndexResult.init; } 9602 9603 string[] sortableFields() { return null; } 9604 string[] searchableFields() { return null; } 9605 9606 struct IndexResult { 9607 Obj[] results; 9608 9609 string[] sortableFields; 9610 9611 string previousPageIdentifier; 9612 string nextPageIdentifier; 9613 string firstPageIdentifier; 9614 string lastPageIdentifier; 9615 9616 int numberOfPages; 9617 } 9618 9619 override string create(scope void delegate() applyChanges) { assert(0); } 9620 override void load(string urlId) { assert(0); } 9621 override void save() { assert(0); } 9622 override void show() { 9623 index(); 9624 } 9625 override void show(string urlId) { 9626 show(); 9627 } 9628 9629 /// Proxy POST requests (create calls) to the child collection 9630 alias PostProxy = Obj; 9631 } 9632 9633 /++ 9634 Serves a REST object, similar to a Ruby on Rails resource. 9635 9636 You put data members in your class. cgi.d will automatically make something out of those. 9637 9638 It will call your constructor with the ID from the URL. This may be null. 9639 It will then populate the data members from the request. 9640 It will then call a method, if present, telling what happened. You don't need to write these! 9641 It finally returns a reply. 9642 9643 Your methods are passed a list of fields it actually set. 9644 9645 The URL mapping - despite my general skepticism of the wisdom - matches up with what most REST 9646 APIs I have used seem to follow. (I REALLY want to put trailing slashes on it though. Works better 9647 with relative linking. But meh.) 9648 9649 GET /items -> index. all values not set. 9650 GET /items/id -> get. only ID will be set, other params ignored. 9651 POST /items -> create. values set as given 9652 PUT /items/id -> replace. values set as given 9653 or POST /items/id with cgi.post["_method"] (thus urlencoded or multipart content-type) set to "PUT" to work around browser/html limitation 9654 a GET with cgi.get["_method"] (in the url) set to "PUT" will render a form. 9655 PATCH /items/id -> update. values set as given, list of changed fields passed 9656 or POST /items/id with cgi.post["_method"] == "PATCH" 9657 DELETE /items/id -> destroy. only ID guaranteed to be set 9658 or POST /items/id with cgi.post["_method"] == "DELETE" 9659 9660 Following the stupid convention, there will never be a trailing slash here, and if it is there, it will 9661 redirect you away from it. 9662 9663 API clients should set the `Accept` HTTP header to application/json or the cgi.get["_format"] = "json" var. 9664 9665 I will also let you change the default, if you must. 9666 9667 // One add-on is validation. You can issue a HTTP GET to a resource with _method = VALIDATE to check potential changes. 9668 9669 You can define sub-resources on your object inside the object. These sub-resources are also REST objects 9670 that follow the same thing. They may be individual resources or collections themselves. 9671 9672 Your class is expected to have at least the following methods: 9673 9674 FIXME: i kinda wanna add a routes object to the initialize call 9675 9676 create 9677 Create returns the new address on success, some code on failure. 9678 show 9679 index 9680 update 9681 remove 9682 9683 You will want to be able to customize the HTTP, HTML, and JSON returns but generally shouldn't have to - the defaults 9684 should usually work. The returned JSON will include a field "href" on all returned objects along with "id". Or omething like that. 9685 9686 Usage of this function will add a dependency on [arsd.dom] and [arsd.jsvar]. 9687 9688 NOT IMPLEMENTED 9689 9690 9691 Really, a collection is a resource with a bunch of subresources. 9692 9693 GET /items 9694 index because it is GET on the top resource 9695 9696 GET /items/foo 9697 item but different than items? 9698 9699 class Items { 9700 9701 } 9702 9703 ... but meh, a collection can be automated. not worth making it 9704 a separate thing, let's look at a real example. Users has many 9705 items and a virtual one, /users/current. 9706 9707 the individual users have properties and two sub-resources: 9708 session, which is just one, and comments, a collection. 9709 9710 class User : RestObject!() { // no parent 9711 int id; 9712 string name; 9713 9714 // the default implementations of the urlId ones is to call load(that_id) then call the arg-less one. 9715 // but you can override them to do it differently. 9716 9717 // any member which is of type RestObject can be linked automatically via href btw. 9718 9719 void show() {} 9720 void show(string urlId) {} // automated! GET of this specific thing 9721 void create() {} // POST on a parent collection - this is called from a collection class after the members are updated 9722 void replace(string urlId) {} // this is the PUT; really, it just updates all fields. 9723 void update(string urlId, string[] fieldList) {} // PATCH, it updates some fields. 9724 void remove(string urlId) {} // DELETE 9725 9726 void load(string urlId) {} // the default implementation of show() populates the id, then 9727 9728 this() {} 9729 9730 mixin Subresource!Session; 9731 mixin Subresource!Comment; 9732 } 9733 9734 class Session : RestObject!() { 9735 // the parent object may not be fully constructed/loaded 9736 this(User parent) {} 9737 9738 } 9739 9740 class Comment : CollectionOf!Comment { 9741 this(User parent) {} 9742 } 9743 9744 class Users : CollectionOf!User { 9745 // but you don't strictly need ANYTHING on a collection; it will just... collect. Implement the subobjects. 9746 void index() {} // GET on this specific thing; just like show really, just different name for the different semantics. 9747 User create() {} // You MAY implement this, but the default is to create a new object, populate it from args, and then call create() on the child 9748 } 9749 9750 +/ 9751 auto serveRestObject(T)(string urlPrefix) { 9752 assert(urlPrefix[0] == '/'); 9753 assert(urlPrefix[$ - 1] != '/', "Do NOT use a trailing slash on REST objects."); 9754 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, immutable void* details) { 9755 string url = cgi.pathInfo[urlPrefix.length .. $]; 9756 9757 if(url.length && url[$ - 1] == '/') { 9758 // remove the final slash... 9759 cgi.setResponseLocation(cgi.scriptName ~ cgi.pathInfo[0 .. $ - 1]); 9760 return true; 9761 } 9762 9763 return restObjectServeHandler!T(cgi, presenter, url); 9764 } 9765 return DispatcherDefinition!internalHandler(urlPrefix, false); 9766 } 9767 9768 /+ 9769 /// Convenience method for serving a collection. It will be named the same 9770 /// as type T, just with an s at the end. If you need any further, just 9771 /// write the class yourself. 9772 auto serveRestCollectionOf(T)(string urlPrefix) { 9773 assert(urlPrefix[0] == '/'); 9774 mixin(`static class `~T.stringof~`s : CollectionOf!(T) {}`); 9775 return serveRestObject!(mixin(T.stringof ~ "s"))(urlPrefix); 9776 } 9777 +/ 9778 9779 bool restObjectServeHandler(T, Presenter)(Cgi cgi, Presenter presenter, string url) { 9780 string urlId = null; 9781 if(url.length && url[0] == '/') { 9782 // asking for a subobject 9783 urlId = url[1 .. $]; 9784 foreach(idx, ch; urlId) { 9785 if(ch == '/') { 9786 urlId = urlId[0 .. idx]; 9787 break; 9788 } 9789 } 9790 } 9791 9792 // FIXME handle other subresources 9793 9794 static if(is(T : CollectionOf!(C), C)) { 9795 if(urlId !is null) { 9796 return restObjectServeHandler!(C, Presenter)(cgi, presenter, url); // FIXME? urlId); 9797 } 9798 } 9799 9800 // FIXME: support precondition failed, if-modified-since, expectation failed, etc. 9801 9802 auto obj = new T(); 9803 obj.toHtmlFromReflection = delegate(t) { 9804 import arsd.dom; 9805 auto div = Element.make("div"); 9806 div.addClass("Dclass_" ~ T.stringof); 9807 div.dataset.url = urlId; 9808 bool first = true; 9809 foreach(idx, memberName; __traits(derivedMembers, T)) 9810 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 9811 if(!first) div.addChild("br"); else first = false; 9812 div.appendChild(presenter.formatReturnValueAsHtml(__traits(getMember, obj, memberName))); 9813 } 9814 return div; 9815 }; 9816 obj.toJsonFromReflection = delegate(t) { 9817 import arsd.jsvar; 9818 var v = var.emptyObject(); 9819 foreach(idx, memberName; __traits(derivedMembers, T)) 9820 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 9821 v[memberName] = __traits(getMember, obj, memberName); 9822 } 9823 return v; 9824 }; 9825 obj.validateFromReflection = delegate(t) { 9826 // FIXME 9827 return ValidationResult.valid; 9828 }; 9829 obj.initialize(cgi); 9830 // FIXME: populate reflection info delegates 9831 9832 9833 // FIXME: I am not happy with this. 9834 switch(urlId) { 9835 case "script.js": 9836 cgi.setResponseContentType("text/javascript"); 9837 cgi.gzipResponse = true; 9838 cgi.write(presenter.script(), true); 9839 return true; 9840 case "style.css": 9841 cgi.setResponseContentType("text/css"); 9842 cgi.gzipResponse = true; 9843 cgi.write(presenter.style(), true); 9844 return true; 9845 default: 9846 // intentionally blank 9847 } 9848 9849 9850 9851 9852 static void applyChangesTemplate(Obj)(Cgi cgi, Obj obj) { 9853 foreach(idx, memberName; __traits(derivedMembers, Obj)) 9854 static if(__traits(compiles, __traits(getMember, obj, memberName).offsetof)) { 9855 __traits(getMember, obj, memberName) = cgi.request(memberName, __traits(getMember, obj, memberName)); 9856 } 9857 } 9858 void applyChanges() { 9859 applyChangesTemplate(cgi, obj); 9860 } 9861 9862 string[] modifiedList; 9863 9864 void writeObject(bool addFormLinks) { 9865 if(cgi.request("format") == "json") { 9866 cgi.setResponseContentType("application/json"); 9867 cgi.write(obj.toJson().toString, true); 9868 } else { 9869 auto container = presenter.htmlContainer(); 9870 if(addFormLinks) { 9871 static if(is(T : CollectionOf!(C), C)) 9872 container.appendHtml(` 9873 <form> 9874 <button type="submit" name="_method" value="POST">Create New</button> 9875 </form> 9876 `); 9877 else 9878 container.appendHtml(` 9879 <form> 9880 <button type="submit" name="_method" value="PATCH">Edit</button> 9881 <button type="submit" name="_method" value="DELETE">Delete</button> 9882 </form> 9883 `); 9884 } 9885 container.appendChild(obj.toHtml()); 9886 cgi.write(container.parentDocument.toString, true); 9887 } 9888 } 9889 9890 // FIXME: I think I need a set type in here.... 9891 // it will be nice to pass sets of members. 9892 9893 try 9894 switch(cgi.requestMethod) { 9895 case Cgi.RequestMethod.GET: 9896 // I could prolly use template this parameters in the implementation above for some reflection stuff. 9897 // sure, it doesn't automatically work in subclasses... but I instantiate here anyway... 9898 9899 // automatic forms here for usable basic auto site from browser. 9900 // even if the format is json, it could actually send out the links and formats, but really there i'ma be meh. 9901 switch(cgi.request("_method", "GET")) { 9902 case "GET": 9903 static if(is(T : CollectionOf!(C), C)) { 9904 auto results = obj.index(); 9905 if(cgi.request("format", "html") == "html") { 9906 auto container = presenter.htmlContainer(); 9907 auto html = presenter.formatReturnValueAsHtml(results.results); 9908 container.appendHtml(` 9909 <form> 9910 <button type="submit" name="_method" value="POST">Create New</button> 9911 </form> 9912 `); 9913 9914 container.appendChild(html); 9915 cgi.write(container.parentDocument.toString, true); 9916 } else { 9917 cgi.setResponseContentType("application/json"); 9918 import arsd.jsvar; 9919 var json = var.emptyArray; 9920 foreach(r; results.results) { 9921 var o = var.emptyObject; 9922 foreach(idx, memberName; __traits(derivedMembers, typeof(r))) 9923 static if(__traits(compiles, __traits(getMember, r, memberName).offsetof)) { 9924 o[memberName] = __traits(getMember, r, memberName); 9925 } 9926 9927 json ~= o; 9928 } 9929 cgi.write(json.toJson(), true); 9930 } 9931 } else { 9932 obj.show(urlId); 9933 writeObject(true); 9934 } 9935 break; 9936 case "PATCH": 9937 obj.load(urlId); 9938 goto case; 9939 case "PUT": 9940 case "POST": 9941 // an editing form for the object 9942 auto container = presenter.htmlContainer(); 9943 static if(__traits(compiles, () { auto o = new obj.PostProxy(); })) { 9944 auto form = (cgi.request("_method") == "POST") ? presenter.createAutomaticFormForObject(new obj.PostProxy()) : presenter.createAutomaticFormForObject(obj); 9945 } else { 9946 auto form = presenter.createAutomaticFormForObject(obj); 9947 } 9948 form.attrs.method = "POST"; 9949 form.setValue("_method", cgi.request("_method", "GET")); 9950 container.appendChild(form); 9951 cgi.write(container.parentDocument.toString(), true); 9952 break; 9953 case "DELETE": 9954 // FIXME: a delete form for the object (can be phrased "are you sure?") 9955 auto container = presenter.htmlContainer(); 9956 container.appendHtml(` 9957 <form method="POST"> 9958 Are you sure you want to delete this item? 9959 <input type="hidden" name="_method" value="DELETE" /> 9960 <input type="submit" value="Yes, Delete It" /> 9961 </form> 9962 9963 `); 9964 cgi.write(container.parentDocument.toString(), true); 9965 break; 9966 default: 9967 cgi.write("bad method\n", true); 9968 } 9969 break; 9970 case Cgi.RequestMethod.POST: 9971 // this is to allow compatibility with HTML forms 9972 switch(cgi.request("_method", "POST")) { 9973 case "PUT": 9974 goto PUT; 9975 case "PATCH": 9976 goto PATCH; 9977 case "DELETE": 9978 goto DELETE; 9979 case "POST": 9980 static if(__traits(compiles, () { auto o = new obj.PostProxy(); })) { 9981 auto p = new obj.PostProxy(); 9982 void specialApplyChanges() { 9983 applyChangesTemplate(cgi, p); 9984 } 9985 string n = p.create(&specialApplyChanges); 9986 } else { 9987 string n = obj.create(&applyChanges); 9988 } 9989 9990 auto newUrl = cgi.scriptName ~ cgi.pathInfo ~ "/" ~ n; 9991 cgi.setResponseLocation(newUrl); 9992 cgi.setResponseStatus("201 Created"); 9993 cgi.write(`The object has been created.`); 9994 break; 9995 default: 9996 cgi.write("bad method\n", true); 9997 } 9998 // FIXME this should be valid on the collection, but not the child.... 9999 // 303 See Other 10000 break; 10001 case Cgi.RequestMethod.PUT: 10002 PUT: 10003 obj.replace(urlId, &applyChanges); 10004 writeObject(false); 10005 break; 10006 case Cgi.RequestMethod.PATCH: 10007 PATCH: 10008 obj.update(urlId, &applyChanges, modifiedList); 10009 writeObject(false); 10010 break; 10011 case Cgi.RequestMethod.DELETE: 10012 DELETE: 10013 obj.remove(urlId); 10014 cgi.setResponseStatus("204 No Content"); 10015 break; 10016 default: 10017 // FIXME: OPTIONS, HEAD 10018 } 10019 catch(Throwable t) { 10020 presenter.presentExceptionAsHtml!(DUMMY)(cgi, t, null); 10021 } 10022 10023 return true; 10024 } 10025 10026 struct DUMMY {} 10027 10028 /+ 10029 struct SetOfFields(T) { 10030 private void[0][string] storage; 10031 void set(string what) { 10032 //storage[what] = 10033 } 10034 void unset(string what) {} 10035 void setAll() {} 10036 void unsetAll() {} 10037 bool isPresent(string what) { return false; } 10038 } 10039 +/ 10040 10041 /+ 10042 enum readonly; 10043 enum hideonindex; 10044 +/ 10045 10046 /++ 10047 Serves a static file. To be used with [dispatcher]. 10048 10049 See_Also: [serveApi], [serveRestObject], [dispatcher], [serveRedirect] 10050 +/ 10051 auto serveStaticFile(string urlPrefix, string filename = null, string contentType = null) { 10052 // https://baus.net/on-tcp_cork/ 10053 // man 2 sendfile 10054 assert(urlPrefix[0] == '/'); 10055 if(filename is null) 10056 filename = urlPrefix[1 .. $]; 10057 if(contentType is null) { 10058 contentType = contentTypeFromFileExtension(filename); 10059 } 10060 10061 static struct DispatcherDetails { 10062 string filename; 10063 string contentType; 10064 } 10065 10066 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 10067 if(details.contentType.indexOf("image/") == 0) 10068 cgi.setCache(true); 10069 cgi.setResponseContentType(details.contentType); 10070 cgi.write(std.file.read(details.filename), true); 10071 return true; 10072 } 10073 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, true, DispatcherDetails(filename, contentType)); 10074 } 10075 10076 string contentTypeFromFileExtension(string filename) { 10077 if(filename.endsWith(".png")) 10078 return "image/png"; 10079 if(filename.endsWith(".svg")) 10080 return "image/svg+xml"; 10081 if(filename.endsWith(".jpg")) 10082 return "image/jpeg"; 10083 if(filename.endsWith(".html")) 10084 return "text/html"; 10085 if(filename.endsWith(".css")) 10086 return "text/css"; 10087 if(filename.endsWith(".js")) 10088 return "application/javascript"; 10089 if(filename.endsWith(".wasm")) 10090 return "application/wasm"; 10091 if(filename.endsWith(".mp3")) 10092 return "audio/mpeg"; 10093 return null; 10094 } 10095 10096 /// This serves a directory full of static files, figuring out the content-types from file extensions. 10097 /// It does not let you to descend into subdirectories (or ascend out of it, of course) 10098 auto serveStaticFileDirectory(string urlPrefix, string directory = null) { 10099 assert(urlPrefix[0] == '/'); 10100 assert(urlPrefix[$-1] == '/'); 10101 10102 static struct DispatcherDetails { 10103 string directory; 10104 } 10105 10106 if(directory is null) 10107 directory = urlPrefix[1 .. $]; 10108 10109 assert(directory[$-1] == '/'); 10110 10111 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 10112 auto file = cgi.pathInfo[urlPrefix.length .. $]; 10113 if(file.indexOf("/") != -1 || file.indexOf("\\") != -1) 10114 return false; 10115 10116 auto contentType = contentTypeFromFileExtension(file); 10117 10118 auto fn = details.directory ~ file; 10119 if(std.file.exists(fn)) { 10120 //if(contentType.indexOf("image/") == 0) 10121 //cgi.setCache(true); 10122 //else if(contentType.indexOf("audio/") == 0) 10123 cgi.setCache(true); 10124 cgi.setResponseContentType(contentType); 10125 cgi.write(std.file.read(fn), true); 10126 return true; 10127 } else { 10128 return false; 10129 } 10130 } 10131 10132 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, false, DispatcherDetails(directory)); 10133 } 10134 10135 /++ 10136 Redirects one url to another 10137 10138 See_Also: [dispatcher], [serveStaticFile] 10139 +/ 10140 auto serveRedirect(string urlPrefix, string redirectTo, int code = 303) { 10141 assert(urlPrefix[0] == '/'); 10142 static struct DispatcherDetails { 10143 string redirectTo; 10144 string code; 10145 } 10146 10147 static bool internalHandler(string urlPrefix, Cgi cgi, Object presenter, DispatcherDetails details) { 10148 cgi.setResponseLocation(details.redirectTo, true, details.code); 10149 return true; 10150 } 10151 10152 10153 return DispatcherDefinition!(internalHandler, DispatcherDetails)(urlPrefix, true, DispatcherDetails(redirectTo, getHttpCodeText(code))); 10154 } 10155 10156 /// Used exclusively with `dispatchTo` 10157 struct DispatcherData(Presenter) { 10158 Cgi cgi; /// You can use this cgi object. 10159 Presenter presenter; /// This is the presenter from top level, and will be forwarded to the sub-dispatcher. 10160 size_t pathInfoStart; /// This is forwarded to the sub-dispatcher. It may be marked private later, or at least read-only. 10161 } 10162 10163 /++ 10164 Dispatches the URL to a specific function. 10165 +/ 10166 auto handleWith(alias handler)(string urlPrefix) { 10167 // cuz I'm too lazy to do it better right now 10168 static class Hack : WebObject { 10169 static import std.traits; 10170 @UrlName("") 10171 auto handle(std.traits.Parameters!handler args) { 10172 return handler(args); 10173 } 10174 } 10175 10176 return urlPrefix.serveApiInternal!Hack; 10177 } 10178 10179 /++ 10180 Dispatches the URL (and anything under it) to another dispatcher function. The function should look something like this: 10181 10182 --- 10183 bool other(DD)(DD dd) { 10184 return dd.dispatcher!( 10185 "/whatever".serveRedirect("/success"), 10186 "/api/".serveApi!MyClass 10187 ); 10188 } 10189 --- 10190 10191 The `DD` in there will be an instance of [DispatcherData] which you can inspect, or forward to another dispatcher 10192 here. It is a template to account for any Presenter type, so you can do compile-time analysis in your presenters. 10193 Or, of course, you could just use the exact type in your own code. 10194 10195 You return true if you handle the given url, or false if not. Just returning the result of [dispatcher] will do a 10196 good job. 10197 10198 10199 +/ 10200 auto dispatchTo(alias handler)(string urlPrefix) { 10201 assert(urlPrefix[0] == '/'); 10202 assert(urlPrefix[$-1] != '/'); 10203 static bool internalHandler(Presenter)(string urlPrefix, Cgi cgi, Presenter presenter, const void* details) { 10204 return handler(DispatcherData!Presenter(cgi, presenter, urlPrefix.length)); 10205 } 10206 10207 return DispatcherDefinition!(internalHandler)(urlPrefix, false); 10208 } 10209 10210 /+ 10211 /++ 10212 See [serveStaticFile] if you want to serve a file off disk. 10213 +/ 10214 auto serveStaticData(string urlPrefix, const(void)[] data, string contentType) { 10215 10216 } 10217 +/ 10218 10219 /++ 10220 A URL dispatcher. 10221 10222 --- 10223 if(cgi.dispatcher!( 10224 "/api/".serveApi!MyApiClass, 10225 "/objects/lol".serveRestObject!MyRestObject, 10226 "/file.js".serveStaticFile, 10227 "/admin/".dispatchTo!adminHandler 10228 )) return; 10229 --- 10230 10231 10232 You define a series of url prefixes followed by handlers. 10233 10234 [dispatchTo] will send the request to another function for handling. 10235 You may want to do different pre- and post- processing there, for example, 10236 an authorization check and different page layout. You can use different 10237 presenters and different function chains. NOT IMPLEMENTED 10238 +/ 10239 template dispatcher(definitions...) { 10240 bool dispatcher(Presenter)(Cgi cgi, Presenter presenterArg = null) { 10241 static if(is(Presenter == typeof(null))) { 10242 static class GenericWebPresenter : WebPresenter!(GenericWebPresenter) {} 10243 auto presenter = new GenericWebPresenter(); 10244 } else 10245 alias presenter = presenterArg; 10246 10247 return dispatcher(DispatcherData!(typeof(presenter))(cgi, presenter, 0)); 10248 } 10249 10250 bool dispatcher(DispatcherData)(DispatcherData dispatcherData) if(!is(DispatcherData : Cgi)) { 10251 // I can prolly make this more efficient later but meh. 10252 foreach(definition; definitions) { 10253 if(definition.rejectFurther) { 10254 if(dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart .. $] == definition.urlPrefix) { 10255 auto ret = definition.handler( 10256 dispatcherData.cgi.pathInfo[0 .. dispatcherData.pathInfoStart + definition.urlPrefix.length], 10257 dispatcherData.cgi, dispatcherData.presenter, definition.details); 10258 if(ret) 10259 return true; 10260 } 10261 } else if( 10262 dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart .. $].startsWith(definition.urlPrefix) && 10263 // cgi.d dispatcher urls must be complete or have a /; 10264 // "foo" -> thing should NOT match "foobar", just "foo" or "foo/thing" 10265 (definition.urlPrefix[$-1] == '/' || (dispatcherData.pathInfoStart + definition.urlPrefix.length) == dispatcherData.cgi.pathInfo.length 10266 || dispatcherData.cgi.pathInfo[dispatcherData.pathInfoStart + definition.urlPrefix.length] == '/') 10267 ) { 10268 auto ret = definition.handler( 10269 dispatcherData.cgi.pathInfo[0 .. dispatcherData.pathInfoStart + definition.urlPrefix.length], 10270 dispatcherData.cgi, dispatcherData.presenter, definition.details); 10271 if(ret) 10272 return true; 10273 } 10274 } 10275 return false; 10276 } 10277 } 10278 10279 }); 10280 10281 private struct StackBuffer { 10282 char[1024] initial = void; 10283 char[] buffer; 10284 size_t position; 10285 10286 this(int a) { 10287 buffer = initial[]; 10288 position = 0; 10289 } 10290 10291 void add(in char[] what) { 10292 if(position + what.length > buffer.length) 10293 buffer.length = position + what.length + 1024; // reallocate with GC to handle special cases 10294 buffer[position .. position + what.length] = what[]; 10295 position += what.length; 10296 } 10297 10298 void add(in char[] w1, in char[] w2, in char[] w3 = null) { 10299 add(w1); 10300 add(w2); 10301 add(w3); 10302 } 10303 10304 void add(long v) { 10305 char[16] buffer = void; 10306 auto pos = buffer.length; 10307 bool negative; 10308 if(v < 0) { 10309 negative = true; 10310 v = -v; 10311 } 10312 do { 10313 buffer[--pos] = cast(char) (v % 10 + '0'); 10314 v /= 10; 10315 } while(v); 10316 10317 if(negative) 10318 buffer[--pos] = '-'; 10319 10320 auto res = buffer[pos .. $]; 10321 10322 add(res[]); 10323 } 10324 10325 char[] get() @nogc { 10326 return buffer[0 .. position]; 10327 } 10328 } 10329 10330 // duplicated in http2.d 10331 private static string getHttpCodeText(int code) pure nothrow @nogc { 10332 switch(code) { 10333 case 200: return "200 OK"; 10334 case 201: return "201 Created"; 10335 case 202: return "202 Accepted"; 10336 case 203: return "203 Non-Authoritative Information"; 10337 case 204: return "204 No Content"; 10338 case 205: return "205 Reset Content"; 10339 // 10340 case 300: return "300 Multiple Choices"; 10341 case 301: return "301 Moved Permanently"; 10342 case 302: return "302 Found"; 10343 case 303: return "303 See Other"; 10344 case 307: return "307 Temporary Redirect"; 10345 case 308: return "308 Permanent Redirect"; 10346 // 10347 // FIXME: add more common 400 ones cgi.d might return too 10348 case 400: return "400 Bad Request"; 10349 case 403: return "403 Forbidden"; 10350 case 404: return "404 Not Found"; 10351 case 405: return "405 Method Not Allowed"; 10352 case 406: return "406 Not Acceptable"; 10353 case 409: return "409 Conflict"; 10354 case 410: return "410 Gone"; 10355 // 10356 case 500: return "500 Internal Server Error"; 10357 case 501: return "501 Not Implemented"; 10358 case 502: return "502 Bad Gateway"; 10359 case 503: return "503 Service Unavailable"; 10360 // 10361 default: assert(0, "Unsupported http code"); 10362 } 10363 } 10364 10365 10366 /+ 10367 /++ 10368 This is the beginnings of my web.d 2.0 - it dispatches web requests to a class object. 10369 10370 It relies on jsvar.d and dom.d. 10371 10372 10373 You can get javascript out of it to call. The generated functions need to look 10374 like 10375 10376 function name(a,b,c,d,e) { 10377 return _call("name", {"realName":a,"sds":b}); 10378 } 10379 10380 And _call returns an object you can call or set up or whatever. 10381 +/ 10382 bool apiDispatcher()(Cgi cgi) { 10383 import arsd.jsvar; 10384 import arsd.dom; 10385 } 10386 +/ 10387 /* 10388 Copyright: Adam D. Ruppe, 2008 - 2021 10389 License: [http://www.boost.org/LICENSE_1_0.txt|Boost License 1.0]. 10390 Authors: Adam D. Ruppe 10391 10392 Copyright Adam D. Ruppe 2008 - 2021. 10393 Distributed under the Boost Software License, Version 1.0. 10394 (See accompanying file LICENSE_1_0.txt or copy at 10395 http://www.boost.org/LICENSE_1_0.txt) 10396 */